{{Quickfixn}} Volume of data

[Ramon de Groot]

if possible, you should include the quickfix/n source code in your 
application (not just the binary) and run a performance profile to see 
where the bottleneck(s) is/are.
in my experience, writing fix messages to disk was never a problem.
however, there are quite a few places where the quickfix/n code can be 
sped up by use of cleverer parsing functions for the various fields 
(datetime, number), I suppose that is what commercial providers do to 
make their implementations faster.




On 23.04.2018 15:59, Kunal CHANGELA wrote:
>
> Instead of writing to disk you could write to an in-memory queue and 
> then have another thread consume that.
>
> I think you want to effectively receive the message, add to your queue 
> and return back….dont spend time in the handler of the message to do work.
>
> Thanks,
>
> CIB_BM_Sign_E_Q
>
> *Kunal Changela*
>
> *Credit Risk and PnL Development*
>
> 	
>
> Group Email: DL CRD IT NY <mailto:crditny at us.bnpparibas.com> (x841 
> 4876, Intl: x721 4876)
>
> 787 7th Avenue, New York, NY 10019
>
> Tel: +1 212 841 2640
>
> Mobile: +1 917 216 8852
>
> kunal.changela at us.bnpparibas.com 
> <mailto:kunal.changela at us.bnpparibas.com>
>
>
> *icône représentant un arbre et symbolisant la nature  Do not print 
> this document unless it is necessary, consider the environment*
>
> *From:*Quickfixn [mailto:quickfixn-bounces at lists.quickfixn.com] *On 
> Behalf Of *Gabe Barwick
> *Sent:* Monday, April 23, 2018 9:47 AM
> *To:* quickfixn at lists.quickfixn.com
> *Subject:* [EXTERNAL] {{Quickfixn}} Volume of data
>
> Hi,
>
> I am receiving a large amount of data (1K+ fix messages per second) 
> from a vendor on my Windows 2008 server.
>
> In OnMessage() I put a return so that I don't actually do anything 
> with the data. All that happens is that QuickFix/N itself writes fix 
> message to SS disk.
>
> My vendor tells me that I am a slow "consumer" at peak times, meaning 
> I am not handling messages fast enough.
>
> What can I do to improve things? As I am not doing anything other than 
> disk-write, am I correct to assume this is the problem?
>
> Should I use a background writing class (on separate thread) instead 
> of QuickFix's
>
> own?
>
> Any thoughts greatly appreciated
>
>
> This message and any attachments (the "message") is intended solely 
> for the addressees and is confidential. If you receive this message in 
> error, please delete it and immediately notify the sender. Any use not 
> in accord with its purpose, any dissemination or disclosure, either 
> whole or partial, is prohibited except formal approval. The internet 
> can not guarantee the integrity of this message. BNP PARIBAS (and its 
> subsidiaries) shall (will) not therefore be liable for the message if 
> modified. Please note that certain functions and services for BNP 
> Paribas may be performed by BNP Paribas RCC, Inc.
>
>
> Unless otherwise provided above, this message was sent by BNP Paribas, 
> or one of its affiliates in Canada, having an office at 1981 McGill 
> College Avenue, Montreal, QC, H3A 2W8, Canada. To the extent this 
> message is being sent from or to Canada, you may unsubscribe from 
> receiving commercial electronic messages by using this link: 
> www.bnpparibas.ca/en/unsubscribe/ 
> <http://www.bnpparibas.ca/en/unsubscribe/>. See www.bnpparibas.ca 
> <http://www.bnpparibas.ca> for more information on BNP Paribas, in Canada.
>
>
>
> _______________________________________________
> Quickfixn mailing list
> Quickfixn at lists.quickfixn.com
> http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com

-- 
Ramon de Groot
CEO Fountainhead Trading AG
Reithystrasse 111
8810 Horgen
Switzerland
+41 43 810 4386

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20180423/39617486/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 11356 bytes
Desc: not available
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20180423/39617486/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.gif
Type: image/gif
Size: 4871 bytes
Desc: not available
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20180423/39617486/attachment-0002.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 652 bytes
Desc: not available
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20180423/39617486/attachment-0002.jpg>