I read NVMEDevice.cc code. However, one thing I am not quite sure of why aio_submit is waiting for all IO finished before it returns. This implies that any AIO thread does the IO in sync mode. 

Looking at the following code path:
aio_submit -> _aio_handle, inside _aio_handle, the code loop and wait if num_running > 0.

Is my reading correct? Is this intentional? If so, why?

Appreciate if someone can help me understand this.

--
-Howard