看到一些資料，說Windows的IOCP后發(fā)制人，比Linux的epoll優(yōu)越很多。那么優(yōu)勢到底在哪里？如何選擇Linux或Windows作為服務器呢？
也看到有資料說，同樣在Linux上，使用reactor機制的網(wǎng)絡庫性能比使用Proactor機制的Asio性能好接近1/5，這個例子可能與題目無關，不過我在文中可能會做一些相關的分析。
系統(tǒng)I/O模型 可分為三類：
- 阻塞型(blocking model)，
- 非阻塞同步型(non-blocking model): "wait until any socket is available to read or write from/to buffer, then call non blocking socket function which returns immediately."
- 以及非阻塞異步型(asynchronous aka. overlapping model): "call a socket function which returns immediately, then wait for its completion, then access the result data object"
IOCP基于非阻塞異步模型，而epoll基于非阻塞同步模型。
參考Hyunjik Bae, A game programmer since 1995說的：
Written 7 Dec 2014
Linux affords epoll which is similar to IOCP. Here are the main differences:
- IOCP is based on asynchronous model, but epoll is based on non-blocking model. Asynchronous aka. overlapping model is "call a socket function which returns immediately, then wait for its completion, then access the result data object". Meanwhile, non blocking model is "wait until any socket is available to read or write from/to buffer, then call non blocking socket function which returns immediately." At last, epoll cannot allow app developers to write best code for zero-copy from/to socket buffer. Accessing much memory read/write is expensive in modern hardware, so this can be a issue.
- IOCP allows multiple thread to wait for completion event of the same IOCP port handle. However, epoll allows only one thread. Of course, calling the wait function (epoll_wait) from several threads works, but it may cause multiple threads awake by the same socket, which is not we want to get. One workaround way is to create epoll handles as many as threads and distribute many socket handles to them.
- As epoll works not infringing any existing non blocking socket functions (and file functions), using epoll and socket together is easier and more intuitive. However, IOCP and especially TCP connecting and accepting functions are tricky and somewhat awkward: check out ConnectEx and AcceptEx in MSDN.
Linux下開發(fā)
參考[4]比較了Windows IOCP和Linux epoll的性能，結論是如果使用Linux，應該使用支持RSS(multi-queue)的NIC，這樣可以達到與IOCP類似的性能。
Linux下Reactor模式和Proactor模式
Boost.Asio為了兼容Windows和Linux，在Linux上用epoll和select去模擬proactor模式，影響了它的效率和實現(xiàn)復雜度。其效率不及使用原生I/O機制的其它實現(xiàn)為Reactor模式的網(wǎng)絡庫。
結論
引用來自參考[3]的話:
Both epoll and IOCP are suitable for, and typically used to write high performance networking servers handling a large number of concurrent connections. However those technologies differ significantly enough to require different event processing code. This difference most likely will make a common implementation of connection/socket class meaningless, as the amount of duplicated code would be minimal. In several implementation I have done an attempt to unify the code resulted in a much less maintainable code comparing to separate implementations, and was always rejected.
Also when porting, it is usually easier to port the IOCP-based code to use epoll than vice versa.
So my suggestion:
- If you need to develop the cross-platform networking server, you should focus on Windows and start with IOCP support. Once it is done, it would be easy to add epoll-based backend.
- Usually it is futile( [?fju:ta?l], 無用的) to implement the single Connection and ConnectionMgr classes. You will end up not only with a whole lot of #ifdef’s but also with different logic. Better create the base ConnectionMgr class and inherit from it. This way you can keep any shared code in the base class, if there’s any.
- Watch out for the scope of your Connection, and make sure you do not delete the object which has read and/or write operations pending.
參考資料
[1] Why doesn't Linux have a system like IOCP or Rio does?, 2014.
[2] 兩種高性能I/O設計模式(Reactor/Proactor)的比較 - 文章 - 伯樂在線
[3] Practical difference between epoll and Windows IO Completion Ports (IOCP)
[4] Windows IOCP vs Linux EPOLL Performance Comparison
