Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Object or DAG API? is there one API to patch multiple links in a single call? can suggest such a feature if not exist yet? #9294

Closed
3 tasks done
tx0c opened this issue Sep 22, 2022 · 6 comments
Labels
kind/enhancement A net-new feature or improvement to an existing feature

Comments

@tx0c
Copy link

tx0c commented Sep 22, 2022

Checklist

  • My issue is specific & actionable.
  • I am not suggesting a protocol enhancement.
  • I have searched on the issue tracker for my issue.

Description

I see from #7936 (comment)

The Object API is a not-well documented portion of the command line API that allows for working with nodes of data serialized to a range of codes. The DAG API was meant to deprecate it, and has existed since 2017.

but

is there a 1-to-1 migrate guide on how to update existing ipfs object commands (and js http-client API calls)? the DAG api looks too low level, can't find how can ipfs object patch add-link ... work with DAG API?

the ipfs object patch add-link ... (and the according js API) just work, but it still can only patch one link at each call,
just wonder is there one API to patch multiple links in a single call?
https://docs.ipfs.tech/reference/kubo/rpc/#api-v0-object-patch-add-link

https://github.com/ipfs/js-ipfs/blob/master/docs/core-api/OBJECT.md#ipfsobjectpatchaddlinkcid-link-options
in js http-client API like this?

// cid is CID of the DAG node created by adding a link
const cid = await ipfs.object.patch.addLink(node, [
  { name: 'some-link', // size: 10?  why do I need to pass it the size? should it be automatically resolved?
    cid: CID.parse('QmPTkMuuL6PD8...RzerB1ZrrBrkSDD')
  },
  { name: 'another-link', // size: 10?  why do I need to pass it the size? should it be automatically resolved?
    cid: CID.parse('another-cid')
  },
  { name: 'another-link2', // size: 10?  why do I need to pass it the size? should it be automatically resolved?
    cid: CID.parse('another-cid2')
  },
])

even with the FILES API, I don't find such an API can do, the example says use this instead, but files cp can still do one link at a time, would be translated to one ipfs call for each new file addition, not cool, and might be slow

$ ipfs files cp /ipfs/QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn /some-dir
$ ipfs files cp /ipfs/Qmayz4F4UzqcAMitTzU4zCSckDofvxstDuj3y7ajsLLEVs /some-dir/added-file.jpg
$ ipfs files cp /ipfs/<another-cid> /some-dir/another-file.jpg
$ ipfs files cp /ipfs/<another-cid2> /some-dir/another-file2.jpg
$ ipfs files stat --hash /some-dir
...

my use case is in an IPFS based blog publishing system:

  1. for all past published articles, each one contains a index.html and some picture assets, published in a ipfs directory,
  2. now want to add some top level aggregation:
    /author-name/
      index.html           <= to be published...
      rss.xml              <= to be published...
      /new-post/           <= to be published...
      /previous-post/...   with a CID of the already published
      /previous-post2/...  with another CID of the already published

you can see, at time of each new post publication, will need to add some new files (top level rss.xml and the new post), and together with old existing CIDs as sub-directories,

Ideally, if the ipfs.addAll can support that mixed use of uploading new files, and existing CIDs would be the best solution:
https://github.com/ipfs/js-ipfs/blob/master/docs/core-api/FILES.md#ipfsaddallsource-options

can this API be made possible? (I believe this can have many other use cases beyond my example of IPFS blog publishing)

ipfs.addAll([
  { path: 'author-blog/index.html', content: 'top index content', },
  { path: 'author-blog/rss.xml', content: 'the updated top level rss', },
  { path: 'author-blog/new-post/index.html', content: 'the new blog post', },
  { path: 'author-blog/new-post/pic1.png', content: 'the new blog post's assets', },
  { path: 'author-blog/new-post/pic2.jpeg', content: 'the new blog post's assets', },

  // ... and can ipfs.addAll support add existing CIDs?
  { path: 'author-blog/previous-post', link: CID.parse('another-existing-cid-from-previous-publication') },
  { path: 'author-blog/previous-post2', link: CID.parse('more-existing-cids') },
  ...
])
@tx0c tx0c added the kind/enhancement A net-new feature or improvement to an existing feature label Sep 22, 2022
@welcome
Copy link

welcome bot commented Sep 22, 2022

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

  • "Priority" labels will show how urgent this is for the team.
  • "Status" labels will show if this is ready to be worked on, blocked, or in progress.
  • "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

@guseggert
Copy link
Contributor

The Object API is deprecated and we plan on removing it soon. To unblock yourself immediately, you can use MFS instead. We're also working on ipfs dag patch which should be usable in this scenario too, see #4782.

@tx0c
Copy link
Author

tx0c commented Sep 22, 2022

is that one super-powerful be possible?

ipfs.addAll([
  { path: 'author-blog/index.html', content: 'top index content', },
  { path: 'author-blog/rss.xml', content: 'the updated top level rss', },
  { path: 'author-blog/new-post/index.html', content: 'the new blog post', },
  { path: 'author-blog/new-post/pic1.png', content: 'the new blog post's assets', },
  { path: 'author-blog/new-post/pic2.jpeg', content: 'the new blog post's assets', },

  // ... and can ipfs.addAll support add existing CIDs?
  { path: 'author-blog/previous-post', link: CID.parse('another-existing-cid-from-previous-publication') },
  { path: 'author-blog/previous-post2', link: CID.parse('more-existing-cids') },
  ...
])

@tx0c
Copy link
Author

tx0c commented Sep 22, 2022

@guseggert oh man, this was closed too fast, at least give it some days discussion?

The Object API is deprecated and we plan on removing it soon. To unblock yourself immediately, you can use MFS instead
I have listed the ipfs files cp APIs but still does not have the wanted API to add many links at one call?

#9301 (comment) hope it can still have a period for a thorough discussion? at least keep it open for a few days?

@tx0c
Copy link
Author

tx0c commented Sep 22, 2022

working on ipfs dag patch which should be usable in this scenario too, see #4782

it was opened in 2018, but for how long should we wait to be able see it in production? for now there's only object.patch.addLink at least work to add 1 link at a time

@guoliu
Copy link

guoliu commented Nov 3, 2022

The Object API is deprecated and we plan on removing it soon. To unblock yourself immediately, you can use MFS instead. We're also working on ipfs dag patch which should be usable in this scenario too, see #4782.

MFS currently cannot replace Object API, since we cannot add an existing CID to an object without copying the remote content to the local machine. Deprecating Object API without ipfs dag patch available is a breaking change without a migration path.

Can this issue be reopened since it has not been resolved?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement A net-new feature or improvement to an existing feature
Projects
None yet
Development

No branches or pull requests

3 participants