Project Donation Addresses:
PeerScript Labs: PScript9dhNxV5xHGwwcjknh9sxe6s4tVX
Visit https://peercoin.chat to talk with the community.
sudo apt-get install build-essential
sudo apt-get install libssl-dev
sudo apt-get install libdb-dev
sudo apt-get install libdb++-dev
sudo apt-get install libboost-all-dev
sudo apt-get install libqrencode-dev
sudo apt-get install miniupnpc
After running into Unicode indexing errors in Python 2.7, a solution was found by installing the Anaconda Python 3.4 distribution. This will also set up your System Environment Variables.
This will be an example that shows you an easy way to capture the ticker data from btc-e.com for all currency pairs. HDF5, also known as Hierarchical Data Format, and Pandas DataFrame will demonstrate to be powerful for quick analysis on large data-sets.
After you have installed the distribution, open the Anaconda Launcher:
Launch an ipython-notebook. A new window should open directed towards localhost:8888/tree or similar. Under files go to the new button and select “Python 3” under the notebooks section.
To begin, import pandas and the time library by doing the following:
Also, set a variable named url to be the BTC-e api ticker url. In this instance we are concerned with capturing data for each pair available on BTC-e.
Next will be the soul of our data capturing, pd.read_json() .
I recommend setting the delay to time.sleep(2). If you read the API information on BTC-e, it states that it refreshes at 0.5Hz.
We simply read the json from the api url into a data-frame named frame and transpose it. Next, we define keylist as the values of the index (pair names for ex. ‘btc_usd’ to be used as the data-frame key); of which is accomplished by setting keylist = frame.index.values.
In the for loop we then separate out each pair with its index set to the series ‘updated’. We then use store.append(‘key’,data) to append each key and data-set to pd.HDFStore(‘filename.h5’).
Once you stop the loop, you can view the data with the follow:
Type any pair combination on BTC-e as the key such as ‘ltc_usd’ or ‘ppc_usd’ to get the logged data.
Here’s a quick example of plotting the data.
Next we will learn about real time plotting using matplotlib!
First off I would like to thank Wes McKinney for his great work compiling together the pandas library over the past years. His book Python for Data Analysis (~$25 on Amazon) is fantastic and a great addition to your home library.
This post will be an introduction to help you better understand the resources and tools available to you in python and the pandas library. If you have been following the MATLAB data capture tutorials you will transition into these python tutorials fairly well.
Our main objective is to capture the JSON data from the BTC-e API and continually store and analyze it to our liking. To accomplish this we will be utilizing Data Frames (Not to be confused with the Data Frames found in “R”) and Hierarchical Indexing.
Now, I started off hacking away in Python 2.7 with the following Libraries/Packages/IDE:
I recommend downloading the following Python(x,y) Package.
It will provide you with the following:
Our main working environments will be the Spyder IDE along with instances of IPython Notebooks. I have found that this is a great combination for testing out new strategies.
Once you have accumulated the necessary resources for this tutorial continue on to Part 1.2. If you have any trouble feel free to leave comments or contact embeddedthought directly.
This next tutorial will guide you through how to get nice, clean, and readable plots out of Matlab. The data that you have fetched with the API and have stored, will be plotted so you can have a view of what everything looks like while double checking that your code is working correctly.
Please note that there are many ways to do what I’m about to show you, this is one way that i find simple and easy to follow.
From this point, we will assume that you have completed the Capturing Data for each pair that you are monitoring. Your code may look similar to mine if you followed the Part 2: Matlab Data Collection. I have added a couple lines of code at the end of each fetching round for user feed back in the console output. This is not necessary but I like it for Debugging and its nice to see where the code is if matlab is working.
Note: My current if statement will allow the Ticker_Grabber to run 40 rounds, there are some issues with the workspace data getting updated in between rounds. We are working on forcing the data to update in the workspace for live plots. (Simulink may be the answer or we might integrate matlab with python.)
Anyways, as you can see the Ticker_Grabber is fetching the order books history for the following pairs: BTC-USD, LTC-USD, LTC-BTC, and PPC-USD. You can add the code for all the other pairs on BTC-e to compare all markets. I will keep it simple for this example but remember that this can be scaled up for all order pairs.
We want to make note of the .mat files that we are creating for each of the pairs. In my case in the btc_usd_orderBook.m will output two .mat files. One for Bidding and one for Asking. If you are not getting two .mat files for each order pair, please visit the Part 2 – Data Capture for more info. For each order pair, you should have two .mat files to work with. In my case I will have a total of 8 .mat files to plot.
For my purposes, I will be plotting the price and volume separate for each asking and bidding strings. You can however get asking and bidding plots on one graph for comparison, but may not be super clear what is going on. Also, keeping the data separate will allow us to use some data manipulation functions later.
Using the figure() command we can group the graphs for each pair and combine them with the subplot to make a nice 2×2 graph. This line of code above is fetching all the values for the asking price of BTC_USD and plotting them against the time stamp of the BTC-e server.
Subplot 2 is very similar to the code of the asking price, but now we are fetching the volume and plotting those values also against the server time. Be sure to make clear labels for your graphs, that way you can really keep track of what is being displayed. Moving right along.
By now you should be getting the hang of what is going on here. Pay attention to the numbering of the subplot function. There is more info on the how the subplot works if you type ‘doc subplot’ in the workspace. For what we are doing this will work just fine. If you have done everything correctly your plots should come out as shown below. You can tweak the color and line art with a little more code. The next pair for plotting will start with the figure(2) and repeat the steps above.
I hope you have found this post interesting and helpful! If you need any assistance please feel free to contact us. Join us in the Mumble server for live trading advice or bot building help!
Look! Embedded Thought now has its own dedicated chat room for trading, trending discussions, bot building help, cryptocoin talk, and anything really! Catch some of the devs in the mumble server that are working on the trading bots or someone like you just trading crypto. Below is the tutorial on how to get inside.
First you will want to head over to Mumble.info <– That is the wiki page, download the correct client for the OS you are running, and install.
After you install the mumble client you will want to create a username and connect to the following server. Note: if you have a btc-e username you can use that and it will make the verification process faster. This process is used to filter out the trolls that may enter the server.
Add a server to connect to and fill in the ip address below. Connect: 188.8.131.52
If you need Real Pro trading advice be sure to get your account verified and join the Traders Corner!
This is for everyone in the BTC-e Troll box that requested a tutorial of my trading strategy. If you have any questions feel free to comment or message me on BTC-e (saeveritt).
Any and all tips are greatly appreciated. BTC:16v9zKaAGiRiqHrcr1BXzYxiSjXqGuFaTA LTC:LfQKBDrbPcEM6ioKRxCx8iXjrAJx4Yvake PPC:PHFgnQaAAiepKfFHL8tpqhPfxrrsCdGV8M
Sorry I have been taking so long to post with updated information.
With the market exploding like it is I have been using my time
to continually refine and develop the bot. This project will be expanding
into a collaboration between myself and other developers. Perhaps it might be
best to wait and provide you all with a walk-through of the finalized bot
since it is ever evolving.
Thank you for reading!
This project will be continued without sectioning it into 5 parts. I have decided to continue forward by presenting the main ideas and algorithms used; coupled with explanations on why they were chosen.
Please check back within the next few weeks to see the progress!
Thank you and happy trading.
Considerations before starting Part 2:
I will not be going in to full detail on every step and I will be assuming you have a basic understanding of MATLAB. Feel free to leave a reply if you have any questions.
Example of recording order-book movement through time:
After Part 2 we will have the functions necessary to start capturing useful data.
Initially, we want to have our data constructed in a way that will be easy for us to use and manage.
There are 3 sub-functions that pertain to the three pairs I am collecting order-book data on.
We will get started by creating those first.
ltcbtc_orderbookMovement does the following:
Every function underlined in blue you will find in the BTC-e Trading API link I referred to in part one. I underlined urlread2 to emphasize that it needs to be in the same folder as all of the BTC-e API files and functions (Which should be the directory you’re working from).
Continuing ltcbtc_orderbookMovement :
During the for loop we are using strcmp to separate the two types of orders, ‘ask’ and ‘bid’.
We are allocating the ask orders and bid order to individual output cell arrays. After doing so, we then use cellfun to remove all empty cells in the array. This is important because we will be concatenating this data with whatever it was previously. If it is your first time running the function and you do not have variables set up, don’t worry. The following rest of this function will take care of that.
Continuing ltcbtc_orderbookMovement :
The double save is redundant and unecessary.
Now that you have an idea on how we want the incoming information structured, you can go ahead and create two more of these function for BTC/USD and LTC/USD. Rename the variables however you please but make sure to check line 34 and change “orderbook=dataset.ltc_btc;” to which ever pair you’re waiting to gather data from.
This process can also be repeated to create the function priceHistory() referenced to at the beginning of this entry. Just change the necessary information to pull from the function realtime_ticker() referenced to in Part 1.
Now that data gathering is out of the way. We can get to the fun stuff!
Any and all tips are greatly appreciated. BTC:16v9zKaAGiRiqHrcr1BXzYxiSjXqGuFaTA LTC:LfQKBDrbPcEM6ioKRxCx8iXjrAJx4Yvake