{{Quickfixn}} Volume of data

[James Riehl]

Your on message should do nothing other than queue processing. If your writing to disk in the onMessage then you’re doing too much. Queue up and process, I recommend blocking collection. 

Sent from my iPhone

> On Apr 23, 2018, at 10:13, Gabe Barwick <barwickhome at hotmail.com> wrote:
> 
> Hi Kunal,
> 
> So you think the QuickFix/N own FIX message logger is too slow even with SSD?
> 
> I can override it easily, but was hoping it was unnecessary - but you think it would be worthwhile?
> 
> 
>  
> From: Quickfixn <quickfixn-bounces at lists.quickfixn.com> on behalf of Kunal CHANGELA <kunal.changela at us.bnpparibas.com>
> Sent: 23 April 2018 13:59
> To: Mailing list for QuickFIX/n
> Subject: Re: {{Quickfixn}} Volume of data
>  
> Instead of writing to disk you could write to an in-memory queue and then have another thread consume that.
> 
> I think you want to effectively receive the message, add to your queue and return back….dont spend time in the handler of the message to do work.
> 
>  
> 
> Thanks,
> 
> 
> 
> Kunal Changela
> 
> Credit Risk and PnL Development
> 
> Group Email: DL CRD IT NY (x841 4876, Intl: x721 4876)
> 
>  
> 
> 787 7th Avenue, New York, NY 10019
> 
> Tel: +1 212 841 2640
> 
> Mobile: +1 917 216 8852
> 
> kunal.changela at us.bnpparibas.com
> 
>  
> 
> <image002.gif>
> 
> <image003.jpg>  Do not print this document unless it is necessary, consider the environment
> 
>  
> 
> From: Quickfixn [mailto:quickfixn-bounces at lists.quickfixn.com] On Behalf Of Gabe Barwick
> Sent: Monday, April 23, 2018 9:47 AM
> To: quickfixn at lists.quickfixn.com
> Subject: [EXTERNAL] {{Quickfixn}} Volume of data
> 
>  
> 
> Hi,
> 
>  
> 
> I am receiving a large amount of data (1K+ fix messages per second) from a vendor on my Windows 2008 server.
> 
>  
> 
> In OnMessage() I put a return so that I don't actually do anything with the data. All that happens is that QuickFix/N itself writes fix message to SS disk.
> 
>  
> 
> My vendor tells me that I am a slow "consumer" at peak times, meaning I am not handling messages fast enough.
> 
>  
> 
> What can I do to improve things? As I am not doing anything other than disk-write, am I correct to assume this is the problem?
> 
>  
> 
> Should I use a background writing class (on separate thread) instead of QuickFix's
> 
> own?
> 
>  
> 
> Any thoughts greatly appreciated
> 
>  
> 
> 
> This message and any attachments (the "message") is intended solely for the addressees and is confidential. If you receive this message in error, please delete it and immediately notify the sender. Any use not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. The internet can not guarantee the integrity of this message. BNP PARIBAS (and its subsidiaries) shall (will) not therefore be liable for the message if modified. Please note that certain functions and services for BNP Paribas may be performed by BNP Paribas RCC, Inc.
> 
> 
> Unless otherwise provided above, this message was sent by BNP Paribas, or one of its affiliates in Canada, having an office at 1981 McGill College Avenue, Montreal, QC, H3A 2W8, Canada. To the extent this message is being sent from or to Canada, you may unsubscribe from receiving commercial electronic messages by using this link: www.bnpparibas.ca/en/unsubscribe/. See www.bnpparibas.ca for more information on BNP Paribas, in Canada.
> 
> _______________________________________________
> Quickfixn mailing list
> Quickfixn at lists.quickfixn.com
> http://lists.quickfixn.com/listinfo.cgi/quickfixn-quickfixn.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.quickfixn.com/pipermail/quickfixn-quickfixn.com/attachments/20180423/0c73e7c8/attachment.htm>