Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

COIN-M Perpetual Futures #10835

Closed
louisswart opened this issue Oct 25, 2024 · 14 comments
Closed

COIN-M Perpetual Futures #10835

louisswart opened this issue Oct 25, 2024 · 14 comments
Labels
Non-spot Non-spot (margin, leverage, futures) related issues and Pull requests

Comments

@louisswart
Copy link

louisswart commented Oct 25, 2024

Firstly I want to thank the contributors and community for this amazing open source project.

With reference to: #7254

Description

I would like to request the addition of COIN-M Perpetual Futures trading for BTC for the "Supported Futures Exchanges".
My personal order of preference would be (but any one of these would be a good start):

  • Binance (I'm currently using)
  • Bybit (I'm currently using)
  • OKX (I'm currently using)
  • Gate.io (have not used yet)

I have attempted to add this myself but realized I have do not have enough understanding of Python nor the internal working of Freqtrade. I think the effort would be more efficiently and accurately executed by the contributors that already have the understanding in this regard.

I recognize the value of time and can understand that receiving a request to implement functionality that might not be deemed required can seem to be an unproductive effort. I am thus willing to sponsor the implementation to the benefit of the project and community.

Reason

I have been trading BTC/USDT for more than 5 years and have come to the realization that at some stages my profit in USDT is negated by the growth of BTC resulting in an effective loss. Being of the mindset to grow BTC holding instead of USD, it holds the following benefits:

  • When you profit in BTC terms, you're accumulating more BTC and not USD that potentially depreciate against BTC thus canceling the profit
  • By keeping profits in BTC rather than USD, you're automatically "reinvesting" into BTC's potential growth
  • This creates a multiplier effect on your overall portfolio value as BTC grows against USD
@stash86
Copy link
Contributor

stash86 commented Oct 25, 2024

The answer is still the same, we have no plan to support it. So either you work on it, or wait for someone else who wants to do it

@robcaulk
Copy link
Member

Hello,

While it may seem easy to "sponsor" a completely new feature. It misses some important considerations:

  • will you sponsor documentation?
  • will you sponsor test writing?
  • will you continue sponsoring monthly maintenance as exchange APIs change (you are asking for 4 new exchanges to be supported here, they will certainly all need individual attention)?
  • will you sponsor bug reports and investigations?
  • who will sponsor discord support of the new feature as users try to use it?

Food for thought,

Rob

@robcaulk robcaulk added the Non-spot Non-spot (margin, leverage, futures) related issues and Pull requests label Oct 25, 2024
@louisswart
Copy link
Author

Thanks for your reply Rob.

I do not think its a completely new feature, as Futures trading is already supported. But I might be completely wrong by not knowing the underlying required functionality, from the response I gather this might be the case.

To Clarify:
From what I can see it might be the "naming format" of the pair being "BTC/USDT:BTC", which is strange because "BTC/USDT:USDT" is recognized. It gives the warning below:
freqtrade.plugins.pairlist.IPairList - WARNING - Pair BTC/USDT:BTC is not compatible with exchange Binance. Removing it from whitelist..

Setting the "stake_currency" to BTC does get the correct balance from the COIN-M trading account (tested on Binance). So it feels so close to working, if the pair was just not removed from the pair list, but then again possibly missing underlying functionality that I'm not aware of.

Response to your "Food for thought":

Documentation

Since Futures trading is already in the documentation it should not require new documentation other than maybe mentioning the support for COIN-M "stake_currency" as supporting "quote currency" for Futures trading.

Test Writing

I see this as part of the sponsored implementation as I assume functionality is mostly accompanied with testing at time of implementation. I'll be able to do final live confirmation testing since I have setups ready on the first 3 mentioned exchanges. After that its up to the community members using the functionality.

Exchange APIs

I can not quite comment on this, but again Futures trading is already supported and thus already maintained. If the addition of this is only related to the naming format it should have a low support impact requirement.

Discord Support and Bug Reports

As I understand it, this is done as a community and the project contributors do not guarantee any functionality. Once the basis of the functionality is in the code base it would enable the community to support and present bug solutions as they arise.

Sponsoring

I think for a contributor familiar with the code structure it should be relatively straight forward, assuming its only related to the naming format and not a completely new trading mechanism.

Sponsoring 3 to 5 work days @ $100/hour would be my guesstimate for this as a beneficial advancing of an already amazing project.

@Axel-CH
Copy link
Contributor

Axel-CH commented Oct 26, 2024

Hello @louisswart,
This is an interesting feature to add, I'm open to work on it once I finished my current active PR.
However even if it's a feature that you sponsor, the main maintainer must validate such implementation, so at the end it's @xmatthias and other member who will validate or not this feature integration.

@xmatthias
Copy link
Member

From what I can see it might be the "naming format" of the pair being "BTC/USDT:BTC", which is strange because "BTC/USDT:USDT" is recognized.

This was changed in ccxt a while ago (years, not months) - and aligned in freqtrade as well - so while having been a reason in 2022, ccxt pair formatting is no longer relevant.

Setting the "stake_currency" to BTC does get the correct balance from the COIN-M trading account (tested on Binance).

that's not actually relevant - while a necessity - this is the part i would expect the least problems.

[...] but again Futures trading is already supported and thus already maintained. If the addition of this is only related to the naming format it should have a low support impact requirement.

That's clearly based on false / wrong believes.
It's WAY more than "just another pair naming scheme" - more like a new mode.
to start with, they have different calculations - even at first glance (binance, liquidationprice - didn't look at this in more detail for others).
While not necessarily relevant for live (exchange-dependant, really) - you'll certainly want to have dry-run support - otherwise you'll start running "blind" (without possibility to backtest or dry-run).

From an exchange perspective, the 2 are completely seperate products, with completely different api endpoints (on binance, /fapi vs. /dapi).
While the endpoints may seem similar - they're not necessarily identical (some are, some ain't) - and can change independent from each other.
This also means that from a ccxt perspective, they may experience different bugs - and thus need to be seen not as "it's just another pair format".

Once the basis of the functionality is in the code base it would enable the community to support and present bug solutions as they arise.

Well that's the dream - but not really how this community works, unfortunately.
I'd like it to be this way - but it's very often not (especially when it comes to issues with exchange communications).
Many users do lack the technical expertiese to actually look such issues - which ain't a problem necessarily.

If it were this simple - why didn't you submit a pull request? (you're also part of the community) (don't take this wrong - it's an example - not everyone is expected to have the necessary skills / knowledge to contribute in code).

That said - it's also not necessarily true that it'll require "monthly maintenance" - though without maintenance, it's pretty likely that it'll just stop working after a bit.

However (and that's the key point) - volunteer support always depends on personal interest.
As such i'm not a fan of having features sponsored.

For an open source project - as maintainer - with every Pull Request we accept, we ultimately take responsibility to maintain it going forward. So while you may be sponsoring the "initial implementer" (let's ignore the effort to review and validate for now) - that implementer may also be a one-off contributor - dumping the maintenance burden (and eventual bugs introduced?) onto the actual maintainers - who may not have an interest (or in this case, possibility) in maintaining such a feature.

and the project contributors do not guarantee any functionality.

Well so you'll sponsor initial implementation - but think it breaks in a month due to some exchange changes (for whatever reason - it'd require changes in freqtrade due to these changes).
If nobody has an interest (or possibility - see restrictions below) in supporting this - we'd simply be removing this support again if we don't get a "community contribution" fixing this within X amount of time (say, 2 weeks?)?

I doubt that's what you're after - but that's what rob hints at with "ongoing support" - and is essentially what might happen.


The bigger problem i see is actually access.
most current (code) contributors (essentially the ones expected to work on this) are from / located in europe - where futures trading is heavily limited (at least on exchanges that do care about regulations - which is most of the ones you're talking about above) - which heavily restricts the possibility to support even existing (futures) features - letalone adding new ones we already know we'll have problems supporting (thank you Europe).

At this point, I don't really see the benefit to the project if i'm honest.

It'll spread already sparse resources trying to support even more markets on the request of an individual user - and won't provide actual benefits to users other than "well i wanna use this instead of that" (as far as i can see, everyone having access to COIN-M futures also has access to USDM futures.

It's still limited / restricted from european users (probably even more so - at least identically) - so what's the benefit to the general freqtrade user exactly? (i wanna accumulate BTC isn't actually a reason, that's your personal desire - sure - but that's not in scope for freqtrade - which is trading).


To summarize - is it possible? sure
is the effort you estimate realistic? can't tell

is it beneficial to the project? not so sure. It'll spread already sparse resources out even more to support an entirely new product which won't actually improve availability for the user (Futures in europe, for example). If it were providing this - it might be different - but it isn't. As such, I think our (limited) resources are better spent on other things.

@louisswart
Copy link
Author

@xmatthias Thanks for taking the time to give such an in depth explanation of what the request would involve. It gave all the necessary perspective regarding the underlying mechanisms that would need to be in place and your position in this regard. I'm sorry to hear about the Futures trading limitation that you experience. My aim with building a strategy has always been to at least outperform "buy and hold" of an asset. With COIN-M "buy and hold" is already achieved (built-in) and any profitable strategy is just cream on top.

I did attempt to look at the code, but quickly found myself lost in a forest of code that I did not have a map of, so I would first need to draw a map to understand the lay of the land. That is when I ignorantly though it might be much more effective for a contributor with an in depth understanding to do the implementation. After your above explanation its also clear that an advance understanding of the API and Futures trading would be required.

@Axel-CH Thanks for you offer to consider my request. From the insights gained I do not think my initial estimate is valid due to the complexity involved. I'm then going to stand down from this in pursuit of another solution.

Again thanks for maintaining and improving Freqtrade.

@xmatthias
Copy link
Member

With COIN-M "buy and hold" is already achieved (built-in) and any profitable strategy is just cream on top.

Already this is a false believe in my opinion.
It may be true in a bull market - but do the same late 2021 to mid 2022 - and you'll have a different opinion - as "buy and hold" was a loser at that point - and having had capital in usdt woud've at least not lost the amount.

I did attempt to look at the code

As said - it was an example - it's pretty clear that people familiar with the codebase will have an easier time - but it's still difficult to determine the exact amount necessary - as only a quick glance showed that quite some details would be rather different.

After your above explanation its also clear that an advance understanding of the API and Futures trading would be required.

About futures trading for sure - API will depend a bit on the exact problems we run into (or not).
I think the complexity is amplified by asking "well i want this - and for 5 exchanges" - which essentially means 5 different new formulas, 5 different API's - with 5 different sets of problems to deal with.

@louisswart
Copy link
Author

louisswart commented Oct 29, 2024

I've been involved with BTC since 2012, so I understand the point you are making. My view is long term, I endured the draw downs and today they are in the past.

My reasoning, stated above, is "to grow BTC holding instead of USD".

Let me illustrate this with a crude example in order to analyze if it is flawed. I know there are multiple aspects that should be taken into account COIN Margin such as funding rates, available margin and more. But these should not be in orders of magnitude more than the ability to generate profit by the strategy. If this is the case there is more behind COIN Margin trading that I'm not aware of yet.

The base assumption is that BTC will increase in value against USD. This assumption is based on the past performance of BTC/USD over a long enough period, thus baring the large draw downs.

Using trading pair BTC/USD
Let say the applied strategy yield is 10% per year on the stake currency.
Using a starting balance of $10,000 running for 5 years (the period) compounding yearly.
Using 1x leverage to compare the result with "Buy and Hold" and reducing costs associated with COIN Margin.

Current BTC price @ $50,000 (for the example)
5 Year BTC price @ $100,000 (this is key to the "base assumption")

Example 1: USD Margin

Here we trade USD against BTC to increase the USD holding by the strategy yield for the period.

Start Balance: $10,000 (0.2 BTC value)
End Balance: $16,105 (0.16105 BTC value)

Yielding a profit of 61% in USD, but a 19% loss in BTC.

Example 2: "Buy and Hold"

Here we just buy BTC and hold it for the period.

Start Balance: 0.2 BTC ($10,000 value)
End Balance: 0.2 BTC ($20,000 value)

Yielding a profit of 100% in USD.

Example 3: COIN Margin

Here we trade BTC against USD to increase the BTC holding by the strategy yield for the period.

Start Balance: 0.2 BTC ($10,000 value)
End Balance: 0.322102 BTC ($32,210 value)

Yielding a profit of 222% in USD and a 61% profit in BTC.

Conclusion

If I have not made a gross miss calculation in example 3 and given my "base assumption", its evident as to which example has the better end result.

So in my view focusing on USD with a strategy that is not able to outperform "Buy and Hold" could results in a future loss. With COIN Margin "Buy and Hold" is already built in.

Feel free to point out any mistakes that might be lurking in my reasoning, I am confidant that example 1 and 2 is correct. Example 3 is new territory for me so I could be wrong.

Exchange API

I was under the impression that the CCXT library shields Freqtrade from the finer details of every exchange API. Assuming then that if one exchange is implemented that only testing the other supported exchanges remains. But from your response this is then not the case. I suppose this is amplified by the complexity of COIN Margin regarding how it is implemented on each exchange.

@dobremha
Copy link

dobremha commented Oct 31, 2024

Hi Louisswart, I think you search in wrong direction. I understand your goal and it make sense, I'm also doing this by myself. But COIN-M have double size funding fee versus normal BTC/USDT perpetual pair, it's not the best ideea to have trades open for a medium/long time. Why pay double money on fee's ?
There are other ways to keep your money in BTC.
You can trade the normal BTC/USDT perpetual pair or other *USDT pairs and keep your money in spot BTC, not in USDT.
In Bybit it's the Portfolio Margin option and in Binance (mandatory for EU users) it's the new BNFCR credit mode.
In both this versions you can keep your trading collateral in BTC, and benefit long term from BTC appreciation.

But ... unfortunately both options are not possible in Freqtrade at this moment.

I think a better approach is to make this two options possible in Freqtrade, one of them would be sufficient for me.

I hope to have more free time towards the end of the year and start working on one of the 2 variant's. I will consult with Matthias on which one, but I am leaning towards Binance, because lately on Bybit we have very low volume on some pairs, having problems filling my orders.

@xmatthias
Copy link
Member

In all honesty - my personal opinion doesn't "really" count if you look at the bot. I would never trade with more than ~5ish leverage - that won't mean we'll restrict the bot to that - so it won't need convincing - and is hence a stale discussion.

Also, you can reason for / against one or the other type of futures - that won't change the fact that we lack access to it (thanks to European Union / our respective countries).

I was under the impression that the CCXT library shields Freqtrade from the finer details of every exchange API. Assuming then that if one exchange is implemented that only testing the other supported exchanges remains. But from your response this is then not the case. I suppose this is amplified by the complexity of COIN Margin regarding how it is implemented on each exchange.

Mostly, that's true.
In detail - we seem to be using some features of ccxt that not many others do - so with every new exchange / mode / ... - we (might) encounter some bugs / problems / things that are not working as they should.
From the exchange (e.g. binance) - it's often not just "a different pair naming" - but a different (although similar) endpoint.
Implicitly this means, ccxt will (potentially) need to implement different handling - which can be correct for one mode - but not (yet) for another mode - which you'll only really see once you start testing "for real".

The devilish detail here is what ccxt doesn't provide - which is calculations - like dry-run calculations - or funding fee calculations (i assume these calculations will mostly be identical across modes, but it needs checking - once per exchange and mode).

In Bybit it's the Portfolio Margin option and in Binance (mandatory for EU users) it's the new BNFCR credit mode.

I'd look at BNFCR mode - but my help in that will be rather limited - as "my" binance doesn't offer BNFCR mode in my country (it's the EU mode - but even within the EU - it's apparently limited to smaller countries - as the bigger ones (france, germany, italy, uk, ...) seem to not be offered this mode).

@dobremha
Copy link

dobremha commented Dec 31, 2024

You can do this now in Binance if you live in an EU country that are allowed to trade Perpetual using the new BNFCR credit mode.

Select any pair you want to trade on normal Perpetual (USDT) or USDC Perpetual ("stake_currency": "USDC"), and keep collateral in BTC, ETH, BNB or USDC.

#10503

Fee structure (funding fee) is 50% less than in COIN-M Perpetual Futures. This is very important, if you keep positions long time open. For me this (big funding fee every 8 hour) was a "no go" in the past, for COIN-M Perpetual Futures.

@xmatthias
Copy link
Member

xmatthias commented Dec 31, 2024

@dobremha i think for european users, it's different again, as we don't have an actual "coin-M" market anymore.
It's hence not really comparable - also because the gains you'll receive will be in BNFCR - not in BTC or USDT/USDC.

@dobremha
Copy link

dobremha commented Dec 31, 2024

Yes, you are right, it's not 100% the same and we don't have it anymore.

But for me the main idea, the main benefit behind this COIN-M issue/request, is having a possibility to trade with a capital that is not USD based. And funding fees on COIN-M was horrible.

Yes, I'm getting profit in BNFCR, but no one is holding me back, to manually exchange every day the gained profit to something else like BTC/ETH/BNB and use it further as collateral in my trading account. This until this bull market will end... after that I will change BNFCR to USDC.

@xmatthias
Copy link
Member

while BNFCR is not what this issue is about - that PR will still work for this (after having done a few more tests).

With margin=cross for binance (available starting with current develop and/or 2025.1), you can keep your balance in BTC (or every other supported currency) and use that as margin - all while still trading USDT or USDC futures markets.
Gains will result in that stake currency - but it's pretty simple to convert this to the desired target currency every few weeks or so.

Supporting COIN-M futures is however not planned.
COIN-M would limit the bot to exactly one trading pair (BTC/USDT:BTC requires BTC as stake - and will not allow ETH/USDT:ETH trading in the same bot - and there's no pairs in the form of ETH/USDT:BTC).

With this limitation - the effort required to implement a new mode is not worth it - hence i'll be closing this as "not planned" - as we don't plan on supporting COIN-M direclty.
However with the very clear pointer that regular cross margin futures will allow you to keep your margin in (almost) any supported Currency or Coin.

@xmatthias xmatthias closed this as not planned Won't fix, can't repro, duplicate, stale Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Non-spot Non-spot (margin, leverage, futures) related issues and Pull requests
Projects
None yet
Development

No branches or pull requests

6 participants