<div><font color="#000000"><font>That's something I can definitely add to the docs.<br></font></font></div><div><font color="#000000"><font><br></font></font></div><div><font color="#000000"><font>Regarding Christian's point, though, if QF is internally violating this guideline, then surely it's worth another look. I cannot guarantee that I will find time for it anytime soon, though. We are always open to patches! :)<br>
</font></font></div><div><font color="#000000"><font><br></font></font></div><br><div class="gmail_quote">On Fri, Jun 8, 2012 at 10:36 AM, Matt Wood <span dir="ltr"><<a href="mailto:mjwood7@gmail.com" target="_blank">mjwood7@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I agree. If creating a different thread for processing methods is what<br>
the developer *should do*, then it should say so in the<br>
documentation/example apps. The example apps (since I last used them)<br>
make no mention of the dangers of spending too much time within the<br>
processing thread. One or the other: documentation or protection.<br>
<span class="HOEnZb"><font color="#888888"><br>
-Matt<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
On Fri, Jun 8, 2012 at 11:03 AM, Christian Jungers<br>
<<a href="mailto:christian.jungers@cm3.com">christian.jungers@cm3.com</a>> wrote:<br>
> Right … the issue is that this relies on the application developer to “do<br>
> the right thing” and keep their receive thread separate from their<br>
> processing thread. But even if the application does use this pattern, the<br>
> problem I’m seeing right now is that the internal QuickFIX.NET library<br>
> doesn’t follow that pattern. Thus, with the large resend request messages<br>
> being automatically / immediately processed on the receive thread, it opens<br>
> the FIX engine up to the possibility of a blocked socket.<br>
><br>
> So all I’m suggesting is to take that pattern and apply it internally to<br>
> QuickFIX as well. That way, whether or not the application developer follows<br>
> this pattern, they will be protected by a robust and scalable QuickFIX<br>
> engine.<br>
><br>
> - Christian<br>
><br>
><br>
><br>
> Christian.Jungers@CM3.com - Chief Technology Officer - Tel<br>
> <a href="tel:877.263.1669%C2%A0x705" value="+18772631669">877.263.1669 x705</a> - Fax <a href="tel:877.263.1669" value="+18772631669">877.263.1669</a><br>
><br>
><br>
><br>
> On Fri, Jun 8, 2012 at 10:56 AM, Thomas Tomiczek <<a href="mailto:t.tomiczek@nettecture.com">t.tomiczek@nettecture.com</a>><br>
> wrote:<br>
>><br>
>> But is this not a bad pattern?<br>
>><br>
>><br>
>><br>
>> The way I handle this is:<br>
>><br>
>><br>
>><br>
>> · Take incoming message<br>
>><br>
>> · Propagate to my own queue, then handle it from there.<br>
>><br>
>><br>
>><br>
>> As a result, I am not even “cracking” messages in the receiving thread,<br>
>> but later in the processing handler. This one runs, as I said, in a<br>
>> separate thread with an in memory queue ;)<br>
>><br>
>><br>
>><br>
>> Yes, it COULD fill up, but that would require a large amount of messages<br>
>> ;)<br>
>><br>
>><br>
>><br>
>> Regards<br>
>><br>
>><br>
>><br>
>> Thomas<br>
>><br>
>><br>
>><br>
>> From: <a href="mailto:quickfixn-bounces@lists.quickfixn.com">quickfixn-bounces@lists.quickfixn.com</a><br>
>> [mailto:<a href="mailto:quickfixn-bounces@lists.quickfixn.com">quickfixn-bounces@lists.quickfixn.com</a>] On Behalf Of Matt Wood<br>
>> Sent: 08 June 2012 16:31<br>
>> To: Mailing list for QuickFIX/n<br>
>> Subject: Re: {{Quickfixn}} Socket Deadlock Issue<br>
>><br>
>><br>
>><br>
>> Christian,<br>
>><br>
>><br>
>><br>
>> Hi, I'm another user and recent contributor of the quickfix/n library. I<br>
>> have encountered this in a related fashion (but triggered differently) and<br>
>> agree with your final paragraph:<br>
>><br>
>><br>
>><br>
>> "However, in the long run, we may need / want to have distinct receive and<br>
>> process threads. As it is, the same thread will block while calling into the<br>
>> actual FIX application for message processing. Should that processing take a<br>
>> long time or involve, for whatever reason, sending a large number of<br>
>> messages ... then we could wind up in a similar problem. Say the FIX<br>
>> application took 2 minutes to process some particular received message (in a<br>
>> blocking synchronous way) ... and during that 2 minutes, the other side of<br>
>> the FIX connection had sent enough messages to fill up it's socket buffer<br>
>> and block. By ensuring that message processing is in a separate thread from<br>
>> the socket reading then we will guarantee (in a far better way) that our<br>
>> socket should never wind up inadvertently blocking the other side. "<br>
>><br>
>><br>
>><br>
>> Some of my processing (mainly database updates) of messages causes a pile<br>
>> up when receiving an exceptionally large number of messages in a short time<br>
>> period. Specifically, I get this at the end of the day (around 5pm) when the<br>
>> system we're connected to propagates a series of IOI messages to mark market<br>
>> offerings as being unavailable (due to traders being forcibly logged out of<br>
>> the system for the day). The pile up manifests itself as "TCP ZeroWindow"<br>
>> messages to be sent from our side telling the foreign system to delay<br>
>> sending further messages. Eventually the delayed TCP messages are sent and<br>
>> received as our system catches up, but are rejected by our fix engine due to<br>
>> the fact that the messages exceed the configured latency (2 minutes!). Of<br>
>> course there is much I can do to alleviate this particular problem, such as<br>
>> creating some sort of queue and threading at a higher place in the app.<br>
>> However, I do feel that a distinct processing thread makes sense and would<br>
>> benefit the overall robustness of the application.<br>
>><br>
>><br>
>><br>
>> -Matt Wood<br>
>><br>
>><br>
>><br>
>><br>
>> _______________________________________________<br>
>> Quickfixn mailing list<br>
>> <a href="mailto:Quickfixn@lists.quickfixn.com">Quickfixn@lists.quickfixn.com</a><br>
>> <a href="http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com" target="_blank">http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com</a><br>
>><br>
><br>
><br>
> _______________________________________________<br>
> Quickfixn mailing list<br>
> <a href="mailto:Quickfixn@lists.quickfixn.com">Quickfixn@lists.quickfixn.com</a><br>
> <a href="http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com" target="_blank">http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com</a><br>
><br>
_______________________________________________<br>
Quickfixn mailing list<br>
<a href="mailto:Quickfixn@lists.quickfixn.com">Quickfixn@lists.quickfixn.com</a><br>
<a href="http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com" target="_blank">http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div><font><span style="background-color:#000000"><span style="background-color:#ffffff">Grant Birchmeier</span></span></font><font><font color="#3333ff"><br>
</font></font></div><div><font><b><font color="#3333ff"><span style="background-color:#ffcc33"><span style="background-color:#ffcc66"><span style="background-color:#ffcc99"><span style="background-color:#ffffff"><span style="background-color:#ffcc00">Connamara Systems, LLC</span></span></span></span></span></font></b></font><br>
</div><div><font><b>Made-To-Measure Trading Solutions.</b></font></div><div><font>Exactly what you need. No more. No less.</font><font><b><font color="#3333ff"><br></font></b></font></div><div><font><a href="http://connamara.com" target="_blank">http://connamara.com</a><br>
</font></div><br>