Zenbot Bitstamp extension #266

tuxitor · 2017-06-11T19:57:52Z

The increase in traffic on the exhanges we have seen lately can be quite a challenge when developing a trading bot. Traffic conguestion, DOS attacks, broken connections and what not. On the exchange side they need tu have mechanisms to conrol the traffic. One measurement the bot are facing is the risk of being blocked with rate limiting. With the use of HTTP requests a lot of connecting, closing and reconnecting are taking place, and it is easy to get into a rate limit situation.

In the interface layer for Bitstamp we have used websockets for getting data. Unfortunately Bitstamp only have implemented two public channel, tickets and orderbook. Luckily it is tickers and orderbook that generate most of the traffic from the exchange. In short it means that we have to use traditional HTTP requests for the private authenticated functions. But use of websockets functions means we can connect once per session and listen for streaming data without the fear of being locked out by a rate limit block.

The Bitstamp interface may still have some bugs and shortcomings, but hopefully such problems can be sorted out.

Introduces a new comand: "zenbot balance <selector>" Sample output: BTC-USD Asset: 9.87654321. Currency: 1.23

…ite a challenge when developing a trading bot. Traffic conguestion, DOS attacks, broken connections and what not. On the exchange side they need tu have mechanisms to conrol the traffic. One measurement the bot are facing is the risk of being blocked with rate limiting. With the use of HTTP requests a lot of connecting, closing and reconnecting are taking place, and it is easy to get into a rate limit situation. In the interface layer for Bitstamp we have used websockets for getting data. Unfortunately Bitstamp only have implemented two public channel, tickets and orderbook. Luckily it is tickers and orderbook that generate most of the traffic from the exchange. In short it means that we have to use traditional HTTP requests for the private authenticated functions. But use of websockets functions means we can connect once per session and listen for streaming data without the fear of being locked out by a rate limit block. The Bitstamp interface may still have some bugs and shortcomings, but hopefully such problems can be sorted out.

Conflicts: commands/balance.js

DeviaVir

the update-products.sh seems to be missing, could you still add this?

Really like the websocket path you took, and it looks great. Just a few nits.

DeviaVir · 2017-06-12T09:34:46Z

extensions/exchanges/bitstamp/exchange.js

+// before the first call for a trade
+// As zenbot dont returns the currency pair
+// before the first trade is requested
+// it ha ben neccessary to get it from 


it ha ben?

DeviaVir · 2017-06-12T09:39:26Z

extensions/exchanges/bitstamp/exchange.js

+      client.balance(null, function (err, body) {
+        body = statusErr(err,body)
+        var balance = {asset: 0, currency: 0}
+	balance.currency = body[opts.currency.toLowerCase() + '_available']


indentation problems start here

DeviaVir · 2017-06-12T09:39:43Z

extensions/exchanges/bitstamp/exchange.js

+      var currencyPair = joinProduct(opts.product_id).toLowerCase()
+      if (typeof opts.order_type === 'undefined' ) {
+	opts.order_type = 'maker'
+	}


indentation

DeviaVir · 2017-06-12T14:27:20Z

@EigilB could you still add an update-products.sh?

tuxitor · 2017-06-12T14:54:29Z

I have not found a function for that on Bitstamp. The products file is done manually, but thanks for asking ;-)

DeviaVir · 2017-06-12T15:00:36Z

@EigilB I see, that's a bit sad, but alright then. Rest looks good to me, I'll leave the PR open for a day and if no one else speaks up let's merge this.

tuxitor · 2017-06-12T21:48:24Z

@DeviaVir, will You put this on hold for a while? During testing I have found an issue that needs some work.

putting on hold

DeviaVir · 2017-06-14T09:23:44Z

@EigilB you might find the changes I made here interesting: https://github.com/carlos8f/zenbot/compare/master...DeviaVir:eigilb-master?expand=1

tuxitor · 2017-06-14T14:30:25Z

@DeviaVir, as a new user of Bitstamp, I think you may find this infornation interesting or at least useful.

The Bitstamp API has a straight forward market order API function, but the limit order has a somewhat peculiar option. This option is called "limit_price" and it is required if we don't use another option. "daily_order" (which is optional).

The 'limit_price' simply means that another order in added in the opposite direction.. It is activated when original order is executed and and the 'limit_price' then gets executed if the the market price reaches the set 'limit price'.

The call won't sell unless we have a value in "limit_price" (tested). It is described like this in the Bitstamp docs:

"If the order gets executed, a new sell order will be placed, with 'limit_price' as its price.", meaning it will place a buy order for the price of "limit_price". That price need to be lower than the sell price and with a corresponding amount.

An example: You set a sell price of 1200. At the same time you set a buy 'limit_price' of 1100. The buy order is activated as soon as the sell order is executed. If the market price goes down to 1100, the 'limit_price' order is executed to buy back the asset.

To use a limit call, I guess some logic to cancel the "limit_price" order is needed.

There is also another option. Instead we can use the "daily_order" option, which means that the same order is automatically canceled at midnight if not executed or canceled before that. Using this option means we need to have two new prototypes where the value of "daily_order" is set to "true" because the existing bitstamp node library is missing this option.

  Bitstamp.prototype.buyDaily = function(market, amount, price, callback) {
    this._post(market, 'buy', callback, {
      amount: amount,
      price: price,
      daily_order: true
    });
  }

And the same for "sellDaily()"

…ange API testing

tuxitor · 2017-06-21T18:37:07Z

@DeviaVir
I think most of the issues are resolved now

DeviaVir · 2017-06-21T18:40:00Z

commands/buy.js

+        } else {
+	  so.order_type = so.taker ? 'taker' : 'maker'
+	  so.order_type === 'taker' ? delete so.taker : delete so.maker
+	}


indenting issue

Hm.. Have tabstops set to 2 without filling with spaces. Expands in other environments

DeviaVir · 2017-06-21T18:40:29Z

commands/sell.js

+        } else {
+	  so.order_type = so.taker ? 'taker' : 'maker'
+	  so.order_type === 'taker' ? delete so.taker : delete so.maker
+	}


indenting issue

DeviaVir · 2017-06-21T18:41:47Z

conf-sample.js

@@ -51,6 +51,13 @@ c.bitfinex.secret = 'YOUR-SECRET'
 // May use 'exchange' or 'trading' wallet balances. However margin trading may not work...read the API documentation.
 c.bitfinex.wallet = 'exchange'

+// to enable Bitfinex trading, enter your API credentials:


Bitfinex => Bitstamp

DeviaVir · 2017-06-21T18:42:49Z

extensions/exchanges/bitstamp/exchange.js

+// before the first trade is requested
+// it has been neccessary to get it from
+// t:he command line arguments
+args.forEach(function(value) {


maybe this should live in a function that is called before it is needed instead of always executed?

I dont quite understand the meaning of this. The internal logic is still a bit "secret". At least I have added comments where this implementation differs from the others. Any suggestions are welcome from a node programmer. This is project is my first attempt to live in the "callback hell".

DeviaVir · 2017-06-21T18:43:34Z

extensions/exchanges/bitstamp/exchange.js

+      price: data.price,
+      side: data.type === 0 ? 'buy' : 'sell'
+    })
+		if (wstrades.length > 30) wstrades.splice(0,10)


weird indenting

DeviaVir · 2017-06-21T18:44:02Z

extensions/exchanges/bitstamp/exchange.js

+  function retry (method, args) {
+    var to = args.wait
+    if (method !== 'getTrades') {
+      console.error(('\nBitstamp API is not answering! unable to call ' + method + ',OB retrying in ' + to + 's').red)


OB retrying, what is OB?

Stupid typo

DeviaVir · 2017-06-21T18:44:38Z

extensions/exchanges/bitstamp/exchange.js

+
+  var exchange = {
+    name: 'bitstamp',
+    historyScan: false,


just verifying, no history for bitstamp available?

Confirmed. From the docs:
"The time interval from which we want the transactions to be returned. Possible values are minute, hour (default) or day."
You can hardly call that history.

DeviaVir · 2017-06-21T18:45:21Z

package.json

@@ -7,7 +7,9 @@
    "zenbot": "./zenbot.sh"
  },
  "dependencies": {
+		"autobahn": "^17.5.2",


weird indenting

DeviaVir · 2017-06-21T18:45:47Z

extensions/exchanges/bitstamp/exchange.js

+          orders['~' + body.id] = body
+          cb(null, body)
+        })
+      } else {


perhaps we can be more explicit here and just say
if (opts.order_type === 'taker') {

Added a comment here to improve clarity

tuxitor · 2017-06-30T17:56:15Z

@carlos8f @DeviaVir
Please tak a look at this. It is probably very interesting for a lot of members of the community:
https://github.com/tuxitor/zenbot/blob/master/docs/Zentalk.md

DeviaVir · 2017-07-03T08:55:34Z

@tuxitor Definitely looks interesting, but it has become quite a mess. I spot a few typo's and wrong indents, but a bit too many to call them all out individually.

It would also be a lot easier if the zentalk and bitstamp code would be separated in different PR's (different branches). Is this something you can still do? Otherwise I'll try to update your code from my end.

tuxitor · 2017-07-03T10:42:17Z

I am far away from home this week, but may have some free spots and be able to do some fix up. Ideally the Zentalk addition should have been done in cooperation with @carlos to find the best approach. In the longer term I think headless networking model is the right direction to go. In general I think it may be possible make an addition like this conditionally included

tuxitor · 2017-07-03T10:45:53Z

I have to add that some help from an experienced node programmer would be much appreciated

tuxitor and others added 19 commits June 6, 2017 00:43

Add files via upload

94d836f

Delete exchanges.md

87613b0

Create exchanges.md

ef20364

Delete exchanges.md

009f885

Create developers.md

8571b6e

Update developers.md

2eddac8

Update developers.md

4f86032

Merge branch 'master' of https://github.com/carlos8f/zenbot

e3f4482

New command "zenbot balance <selector>"

4b54f3f

Introduces a new comand: "zenbot balance <selector>" Sample output: BTC-USD Asset: 9.87654321. Currency: 1.23

New command "zenbot balance <selector>"

b23a912

Merge branch 'master' of https://github.com/carlos8f/zenbot

671459a

Resolved merge conflict by incorporating both suggestions.

685ec0a

Merge branch 'master' into master

c2a94e3

Merge branch 'master' of https://github.com/eigilb/zenbot

386130d

Conflicts: commands/balance.js

Updated conf-sample.js

0f6602c

Small changes in exchange.js

4d87e08

Small change in conf-sample.js

4f0000f

Small insert a LF in gront of error messages

c9dbeec

DeviaVir suggested changes Jun 12, 2017

View reviewed changes

Eigil Bjorgum added 2 commits June 12, 2017 16:06

Done some cosmetic changes

fc1dc4e

Merge branch 'master' of https://github.com/carlos8f/zenbot

338be24

DeviaVir previously approved these changes Jun 12, 2017

View reviewed changes

Eigil Bjorgum added 5 commits June 17, 2017 12:06

Prepare buy/sell for using market order and make it suitable for exch…

c2e5140

…ange API testing

Small changes

b8ac748

Merge branch 'master' of https://github.com/carlos8f/zenbot

4c4b000

Merge branch 'master' of https://github.com/carlos8f/zenbot

1c60673

Changes to package.json and exchange.js

6099841

Delete syncfromCarlos.sh

ad42640

DeviaVir reviewed Jun 21, 2017

View reviewed changes

Eigil Bjorgum and others added 7 commits June 21, 2017 21:47

Fixed indenting

8919ca4

Fix for communication hiccups

13bb7f6

Fix the 5 figures issue in Bitfinex

d319c1c

Fix the fix for the Bitfinex 5 figure limit issue

2b93e48

Merge branch 'master' of https://github.com/carlos8f/zenbot

9670841

Overview of a Zenbot websocket agent

503f4dd

Adding program and some file updates for the Zentalk concept

5b0c01e

Removed port setting from cmd line options in favor of auto grab

568ba50

tuxitor closed this Jul 5, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Zenbot Bitstamp extension #266

Zenbot Bitstamp extension #266

tuxitor commented Jun 11, 2017

DeviaVir left a comment

DeviaVir Jun 12, 2017

DeviaVir Jun 12, 2017

DeviaVir Jun 12, 2017

DeviaVir commented Jun 12, 2017

tuxitor commented Jun 12, 2017

DeviaVir commented Jun 12, 2017

tuxitor commented Jun 12, 2017

DeviaVir commented Jun 14, 2017

tuxitor commented Jun 14, 2017

tuxitor commented Jun 21, 2017 •

edited

Loading

DeviaVir Jun 21, 2017

tuxitor Jun 21, 2017

DeviaVir Jun 21, 2017

DeviaVir Jun 21, 2017

DeviaVir Jun 21, 2017

tuxitor Jun 21, 2017

DeviaVir Jun 21, 2017

DeviaVir Jun 21, 2017

tuxitor Jun 21, 2017

DeviaVir Jun 21, 2017

tuxitor Jun 21, 2017 •

edited

Loading

DeviaVir Jun 21, 2017

DeviaVir Jun 21, 2017

tuxitor Jun 21, 2017

tuxitor commented Jun 30, 2017 •

edited

Loading

DeviaVir commented Jul 3, 2017

tuxitor commented Jul 3, 2017

tuxitor commented Jul 3, 2017

Zenbot Bitstamp extension #266

Zenbot Bitstamp extension #266

Conversation

tuxitor commented Jun 11, 2017

DeviaVir left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DeviaVir commented Jun 12, 2017

tuxitor commented Jun 12, 2017

DeviaVir commented Jun 12, 2017

tuxitor commented Jun 12, 2017

DeviaVir commented Jun 14, 2017

tuxitor commented Jun 14, 2017

tuxitor commented Jun 21, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tuxitor Jun 21, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tuxitor commented Jun 30, 2017 • edited Loading

DeviaVir commented Jul 3, 2017

tuxitor commented Jul 3, 2017

tuxitor commented Jul 3, 2017

tuxitor commented Jun 21, 2017 •

edited

Loading

tuxitor Jun 21, 2017 •

edited

Loading

tuxitor commented Jun 30, 2017 •

edited

Loading