Skip to content

Instagram scraper search hashtags include protected tag endpoints

Notifications You must be signed in to change notification settings

ferrybrouwer/instagram-searchtags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

21 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Instagram search hashtags

npm version GitHub release npm

Why this package?

The API of instagram limits search for hashtags by authorized user tags only with a limit of 20 images. The API also has a rate limit.

Before we could easily fetch all tags with the endpoints /explore/tags. Instagram recently authorized these endpoints. Therefore it will not work for third party apps.

Instead of using the API, this package scrapes data without any limits. It uses phantomjs to get the data.

Install

npm install instagram-searchtags --save

Usage

Add the environment variable DEBUG=instagramsearchtags to run with debug information in the console, example:

DEBUG=instagramsearchtags node some-script.js

Basic example: fetching 10 hashtag #dog nodes

const InstagramSearchTags = require('instagram-searchtags')

// Create instance with your credentials
const searchTags = new InstagramSearchTags({
  username: 'instagram-username-or-email',
  password: 'xxx',
})

// Login Instagram with credentials
searchTags.login()
  .then(() => {

    // Create #dog tag
    const tag = searchTags.createTag('dog')

    // Fetch 10 latest nodes
    return tag.fetchNodes(10)

  })
  .then((nodes) => {

    // ... do something cool with nodes

    // close connection
    searchTags.close()

  })
  .catch((err) => {

    // close connection
    searchTags.close()

    console.error(`Error: ${err.message}`)

  })

See other examples:

Let's break it down in some basic steps:

  1. Create an instance of InstagramSearchTags with your credentials
  2. Login with these credentials with the method login(), this method returns a Promise.
  3. After you've successfully logged in, you will be able to create tags with the method createTag(). This method returns a Tag instance.
  4. With the Tag instance you can invoke several fetch methods, such as fetchNodes() which returns a Promise with the resolved nodes.
  5. Close the PhantomJS connection with the close() method.

Documentation

InstagramSearchTags Class

The InstagramSearchTags Class is the main class to construct and destruct searching Instagram hashtags.

  • Create connection:

     const searchTags = new InstagramSearchTags({
       username: 'instagram-username-or-email',
       password: 'xxx',
     })
  • Create Tag instance:

    Method structure (pseudo code):
    InstagramSearchTags.createTag(:String): Tag

     const dogTag = searchTags.createTag('dog')
  • Close connection:

    Method structure (pseudo code):
    InstagramSearchTags.close(): Promise.<void>

     await searchTags.close()

Tag Class

The Tag Class is used to create tags and fetch data from it.

  • Fetching single page:

    Method structure (pseudo code):
    Tag.fetchPage(maxId:String): Promise.<Page>

    This method fetch a single page object including top_posts etc.
    The parameter maxId is the hash for the page to fetch (this is optional).

     const page = await tag.fetchPage()
  • Fetching next page:

    Method structure (pseudo code):
    Tag.fetchNextPage(): Promise.<Page>

    This method fetch the next page. Useful for manually iterate through pages

     const page = await tag.fetchPage()
    
     if (page.hasNextPage()) {
       const nextPage = await tag.fetchNextPage()
     }
  • Fetching nodes:

    Method structure (pseudo code):
    Tag.fetchNodes(maxNodes:Number): Promise.<Array>

    This method fetch media nodes with a maximum. Internal it uses fetchPage and fetchNextPage recursively till the nodes Array is constructed.

     const nodes = await tag.fetchNodes(20)
  • Download thumbnail images:

    Method structure (pseudo code):
    Tag.downloadNodeThumbnailImages(destinationDirectory:String, maxNodes:Number): Promise.<void>

    This method fetch media nodes with a maximum maxNodes and download image to destinationDirectory. It uses the property thumbnail_src from a single node.

     const nodes = await tag.downloadNodeThumbnailImages('./images', 20)
  • Download display images:

    Method structure (pseudo code):
    Tag.downloadNodeDisplayImages(destinationDirectory:String, maxNodes:Number): Promise.<void>

    This method fetch media nodes with a maximum maxNodes and download image to destinationDirectory. It uses the property display_src from a single node.

     const nodes = await tag.downloadNodeDisplayImages('./images', 20)
  • Get the total amount of media nodes found for given tag query.:

    Method structure (pseudo code):
    Tag.getTotalCount(): Promise.<Number>

     const totalCount = await tag.getTotalCount()
  • Get state if has next page:

    Method structure (pseudo code):
    Tag.hasNextPage(): Boolean

     const hasNextPage = tag.hasNextPage()

Page Class

The Page Class is generated by Tag instances. It provide an interface of fetching data. This class represent a single Instagram query /explore/tags/${tag}?__a=1 entry.

  • Get state if has next page:

    Method structure (pseudo code):
    page.hasNextPage(): Boolean

     const hasNextPage = page.hasNextPage()
  • Get next page hash (maxId):

    Method structure (pseudo code):
    page.getNextPageMaxId(): String|Boolean.<false>

    Get the hash maxId for the next page. Returns false when there is no next page found.

     const maxId = page.getNextPageMaxId()
  • Get page nodes:

    Method structure (pseudo code):
    page.getNodes(): Array

    Returns all media nodes of page

     const nodes = page.getNodes()
  • Get total result count of tag:

    Method structure (pseudo code):
    page.getTotalCount(): Number

    The result of this method is the same as for Tag.prototype.getTotalCount()

     const totalNodeCount = page.getTotalCount()

About

Instagram scraper search hashtags include protected tag endpoints

Resources

Stars

Watchers

Forks

Packages

No packages published