<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">In case you are using FileLog: <br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Both the message log and the event log have their StreamWriter.AutoFlush set to "true". The source can be found at
<a class="moz-txt-link-freetext" href="https://github.com/connamara/quickfixn/blob/master/QuickFIXn/FileLog.cs#L41">
https://github.com/connamara/quickfixn/blob/master/QuickFIXn/FileLog.cs#L41</a><br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">This may hurt performance, because after every single write to the logs, FileStream.Flush(flushToDisk: false) will be called internally. (Which however should not be as bad as flushToDisk = true)</div>
<div class="moz-cite-prefix">What you could do is as a test: Clone QuickFixN's FileLog (and you also have to clone FileLogFactory to be able to use it) into your own project and turn off AutoFlush there. If that does in fact remove the latency spikes you are
seeing, then you can have a look at how to address the frequent flushing. There are a couple of ways to do flushing in a less performance impacting, yet low-latency way.<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Eugen</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">On 2019/05/15 4:03, Grant Birchmeier wrote:<br>
</div>
<blockquote type="cite" cite="mid:CAMq4opPYig_RWWsDXCpR2oFO8Xfh0Y-0wDwK7nY2AtK7RwcUFA@mail.gmail.com">
<div dir="ltr">
<div class="gmail_default" style="font-size:small;color:#000000">I won't pretend I have special knowledge here; I've never looked at logging performance before in QF/n. It could be Windows file IO, or maybe there's something unintentionally silly happening
in QF's internals.</div>
<div class="gmail_default" style="font-size:small;color:#000000"><br>
</div>
<div class="gmail_default" style="font-size:small;color:#000000">To investigate, you could write your own LogFactory implementation and integrate some profiling into it. The interface is pretty simple.</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, May 14, 2019 at 1:55 PM xmoon 2000 <<a href="mailto:xmoon2000@googlemail.com" moz-do-not-send="true">xmoon2000@googlemail.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Grant,<br>
<br>
In your opinion, should QuickFix be able to write to its own log<br>
without causing latencies, if as in my code snippet, I do nothing<br>
except return from FromApp WITHOUT calling<br>
Crack.<br>
<br>
For approx 20m messages per day?<br>
<br>
On Tue, 14 May 2019 at 15:42, Grant Birchmeier<br>
<<a href="mailto:gbirchmeier@connamara.com" target="_blank" moz-do-not-send="true">gbirchmeier@connamara.com</a>> wrote:<br>
><br>
> Is this latency causing performance problems other than the logging delay? Is the delay constant, or does it only occur when there's a huge influx of messages? (Would it get worse than 500ms if the high-traffic continued?)<br>
><br>
><br>
> On Tue, May 14, 2019 at 9:14 AM xmoon 2000 <<a href="mailto:xmoon2000@googlemail.com" target="_blank" moz-do-not-send="true">xmoon2000@googlemail.com</a>> wrote:<br>
>><br>
>> Hi,<br>
>><br>
>> I get about 20 million records via QuickFix .NET in c# per day.<br>
>><br>
>> I have done a pcapture and can see that the tag52 and WireShark times<br>
>> are close - so latency is not from transmission.<br>
>><br>
>> In my app I have stripped everything out (see FromApp below).So that I<br>
>> do nothing with the data - I just allow QuickFix engine to write fix<br>
>> messages to log on my SSD.<br>
>><br>
>> BUT I can see latencies of 500ms between QuickFix log time and tag52 time.<br>
>><br>
>> What might be causing this? Is writing to SSD so slow? Or is it<br>
>> because several other apps are reading this log file?<br>
>><br>
>> public void FromApp(QuickFix.Message msg, SessionID sessionID)<br>
>> {<br>
>> return;<br>
>><br>
>> try<br>
>> {<br>
>> Crack(msg, sessionID);<br>
>> }<br>
>> catch (Exception ex)<br>
>> {<br>
>> Console.WriteLine("==Cracker exception==");<br>
>> Console.WriteLine(ex.ToString());<br>
>> Console.WriteLine(ex.StackTrace);<br>
>> }<br>
>> }<br>
>> _______________________________________________<br>
>> Quickfixn mailing list<br>
>> <a href="mailto:Quickfixn@lists.quickfixn.com" target="_blank" moz-do-not-send="true">
Quickfixn@lists.quickfixn.com</a><br>
>> <a href="http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com" rel="noreferrer" target="_blank" moz-do-not-send="true">
http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com</a><br>
><br>
><br>
><br>
> --<br>
> Grant Birchmeier<br>
> Connamara Systems, LLC<br>
> Made-To-Measure Trading Solutions.<br>
> Exactly what you need. No more. No less.<br>
> <a href="http://connamara.com" rel="noreferrer" target="_blank" moz-do-not-send="true">
http://connamara.com</a><br>
> _______________________________________________<br>
> Quickfixn mailing list<br>
> <a href="mailto:Quickfixn@lists.quickfixn.com" target="_blank" moz-do-not-send="true">
Quickfixn@lists.quickfixn.com</a><br>
> <a href="http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com" rel="noreferrer" target="_blank" moz-do-not-send="true">
http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com</a><br>
_______________________________________________<br>
Quickfixn mailing list<br>
<a href="mailto:Quickfixn@lists.quickfixn.com" target="_blank" moz-do-not-send="true">Quickfixn@lists.quickfixn.com</a><br>
<a href="http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com" rel="noreferrer" target="_blank" moz-do-not-send="true">http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com</a><br>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr" class="gmail_signature">
<div><font size="2"><span style="background-color:rgb(0,0,0)"><span style="background-color:rgb(255,255,255)">Grant Birchmeier</span></span></font><font size="2"><font color="#3333ff"><br>
</font></font></div>
<div><font size="2"><b><font color="#3333ff"><span style="background-color:rgb(255,204,51)"><span style="background-color:rgb(255,204,102)"><span style="background-color:rgb(255,204,153)"><span style="background-color:rgb(255,255,255)"><span style="background-color:rgb(255,204,0)">Connamara
Systems, LLC</span></span></span></span></span></font></b></font><br>
</div>
<div><font size="2"><b>Made-To-Measure Trading Solutions.</b></font></div>
<div><font size="2">Exactly what you need. No more. No less.</font><font size="2"><b><font color="#3333ff"><br>
</font></b></font></div>
<div><font size="2"><a href="http://connamara.com" target="_blank" moz-do-not-send="true">http://connamara.com</a><br>
</font></div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Quickfixn mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Quickfixn@lists.quickfixn.com">Quickfixn@lists.quickfixn.com</a>
<a class="moz-txt-link-freetext" href="http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com">http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com</a>
</pre>
</blockquote>
</body>
</html>