Disclaimer: This is for Educational purposes only. I am not responsible for how you or anyone else uses the following information.
Dilemma: Need real-time streaming quotes to load into my own stock analyzer application so that I can back test strategies. There are a few services out there that will give you streaming quotes for free, but they are delayed by about 15 to 20 minutes on average. To get real-time quotes you would need to sign up to receive data from a provider such as ESignal, IQFeed, Gain, OpenTick ($1/month for real-time quotes), or Yahoo. However if you already have an online broker, you are able to view these quotes via your broker’s website. Usually the broker streams the live quotes through a java applet. I want to use the data where I want to use it and how I want to use it. I would like to use this data feed and provide it to my application, which is written in C#, so that it can analyze the market as it is changes in real time.
Solution: You will need the following tools/resources: an online broker, a packet capturing tool (I use WinPcap), a packet sniffer (I used NetworkActiv PIAFCTM 2.2, but you can use any, such as: WireShark or Ethereal), and SharpPcap (C# wrapper for WinPcap) if you’re going to extract the data using C#, if you use python, as I sometimes do, try pypcap.
Once you have these tools in your arsenal you are ready to begin.
- Install the applications necessary listed above.
- You should stop all network activity on your computer. This means close your AIM application, online games, and torrent applications.
- Open your browser and go to your broker’s website, login, and start the live streaming data feed.
- Start your packet sniffer and take a look at what port the data is coming from, to get the right port you will have to look through the network traffic and see if you can find any ticker data, if you look at the port that it is coming from and take a note of it.
- Start up your C# editor and create a new project. Use the SharpPcap dll as a reference and create your device listener. Once you have the right network adapter selected set the device to filter the port that the data is coming in from. Next write the packet data out to the console. Remember: The packet data is a byte array so properly convert the byte array to ASCII using: System.Text.Encoding.ASCII.GetString(packet.Data). This will properly output the data.
- Now that we are receiving the ticker data, we can use regular expressions to extract the relevant data from the application-specific formated protocol.
- Create a formatted datafeed service for your application. Have the data passed via a network port. Store the ticker data into a database so that your application can access historical data whenever it needs it.
Notes: I found that some online brokers such as TD Ameritrade actually encrypt their network traffic data to the applet, so it is harder to decrypt, however I am sure that the security key is lying somewhere in the code or on the computer. There are some brokers such as OptionsXpress (Free signup and no minimum deposit) that don’t encrypt their data as they send it to their applet, and therefore the data is flying around in plain text, therefore it is much easier to strip that data using a packet sniffing utility.
Comments: I would recommend getting a data feed via a company like OpenTick as they offer it for cheap prices a month. Another good option I have found is use a broker that offers that service to you for free, such as Interactive Brokers. Interactive Brokers has an API to allow you to use real-time data to build your own automated trading applications, you can use their API in C++, C#, and Java.
Leave comments if you’d like to see screen shots or want me to post my code.