[Feature Request] add Cookie option to pass Cloudflare protection

[erickyun]

Hi, I launched bot and when I started bot and clicked on "Random Manga", it didn't responed, checked the log and it said: Werror [Error]: Handling callback_query: Handling Random:next button: Getting random manga: Couldn't find manga with such tags looks like it can't connect to manga source or something like that.

Cause of this problem is Cloudflare protection.

I don't want to solve captchas, and using headless browser won't help with number of requests to big bots.

But If you want to run your own instance, you can pass Cloudflare protection manually and set Cookie header in requests. If you want me to add Cookie option to make this easier - let me know. It will fix the issue till cookie expires.

Hi,

Firstly I want to thank you for your great project sir.👍

I'm having the same problem as you mentioned above Here and I would appreciate it if Could you add the Cookie option you were talking about?

[sleroq]

Yeah, I'll try to do something today

[erickyun]

Yeah, I'll try to do something today

Thanks appreciate it. 🙂

[sleroq]

Working on ability to add other sources for doijins and choose which one to use per-user in e-hentai branch.

In the same update I'm going to add cookies options. I have not decided on how exactly to implement it.
Should cookies be per-user, so anyone could set them in telegram? Or make it so only bot-owner can set / change them.

[erickyun]

Working on ability to add other sources for doijins and choose which one to use per-user in e-hentai branch.

That would be great. 👍

In the same update I'm going to add cookies options. I have not decided on how exactly to implement it.

How about we could reply to cookies.txt file (from the get cookie extension) that bot owner did send with a cmd like /cookie and then cookie file would be renewd

Should cookies be per-user, so anyone could set them in telegram? Or make it so only bot-owner can set / change them.

I would appreciate it if only bot-owner could send it sir. We never know what users may send?
Also private usage would be nice so that only bot owner could use the bot.

[trueimmortal]

I have been messing about with integrating nhentai to my exhentai/ehentai telegram bot and it's not exactly as easy as a 'Cookie' and off you go.

From what i have seen during testing not only does the user agent needs to match the one you got the cookies with, but the IP/Region also needs to match. So this pretty much means you need to host the Bot on the same network that you got the cookies from at the minimum. Another option would be to use something like FlareSolverr .

On my local dev setup i had it working , but I host my stuff somewhere else that's when I hit that particular wall. At the end of the day Nhentai mostly reposts what exhentai has, so i abandoned the idea of having Nhentai on my bot.

[erickyun]

I have been messing about with integrating nhentai to my exhentai/ehentai telegram bot and it's not exactly as easy as a 'Cookie' and off you go.

From what i have seen during testing not only does the user agent needs to match the one you got the cookies with, but the IP/Region also needs to match. So this pretty much means you need to host the Bot on the same network that you got the cookies from at the minimum. Another option would be to use something like FlareSolverr .

So if were to use the bot on a vps, the bot owner could only send the right cookies with a vpn that is setup on the vps right? Since we need the same ip for the cookie to work?

On my local dev setup i had it working , but I host my stuff somewhere else that's when I hit that particular wall. At the end of the day Nhentai mostly reposts what exhentai has, so i abandoned the idea of having Nhentai on my bot.

Could you maybe please not abandon but add Nhentai in a different branch for those who want it to use the bot for private usage? İf it is possible sir please?

[trueimmortal]

So if were to use the bot on a vps, the bot owner could only send the right cookies with a vpn that is setup on the vps right? Since we need the same ip for the cookie to work?

Hmm, i suppose that could also work yes. As long as the cookies are used from the same place/ip they were issued at.

Could you maybe please not abandon but add Nhentai in a different branch for those who want it to use the bot for private usage? İf it is possible sir please?

I am not part of this project, i was just following it's progress.

[erickyun]

So if were to use the bot on a vps, the bot owner could only send the right cookies with a vpn that is setup on the vps right? Since we need the same ip for the cookie to work?

Hmm, i suppose that could also work yes. As long as the cookies are used from the same place/ip they were issued at.

Could you maybe please not abandon but add Nhentai in a different branch for those who want it to use the bot for private usage? İf it is possible sir please?

I am not part of this project, i was just following it's progress.

O my bad sorry about that sir.
Do you plan to make your bot public btw?

[trueimmortal]

@erickyun

Altough this isn't the place to discuss this. Eventually i am. Right now i have a complete stack for it. Bot / Api / Web front end. But since this might be too complicated for most people to setup im reworking my bot so it can be used standalone.

[sleroq]

Hi @trueimmortal, I appreciate your suggestion, I think I have enough RAM, so I will try to install FlareSolverr on my vps.

For now, I’m using a mirror site of nhentai. What puzzles me is why they have cloudflare protection even for the /api/ route.

[trueimmortal]

For now, I’m using a mirror site of nhentai. What puzzles me is why they have cloudflare protection even for the /api/ route.

Haven't got a clue, they got tired of being scraped probably, they don't get any revenue that way. Yeah i also saw you updated the source for an alternative. You also cut down on a lot of features it seems? I still post my stuff to telegraph for now but it can cause issues with larger galleries. I have a setting on it where i cap it at 75 currently. I had users trying to fetch 1000 images galleries lmao

[jodu75]

Hello if you would like to make the bot work with your own cookies you can edit the file nhentai.js like this :

const cookie = 'YOUR COOKIE'
const ua = 'YOUR USER AGENT'

var __importDefault = (this && this.__importDefault) || function (mod) {
    return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.getIdFromUrl = void 0;
const got_1 = __importDefault(require("got"));
const cheerio_1 = __importDefault(require("cheerio"));
class nHentai {
    static async getDoujin(identifier) {
        if (!identifier) {
            throw Error('You have to specify id');
        }
        let response;
        try {
            response = await (0, got_1.default)(`https://nhentai.net/g/${identifier}/`, {
            headers: {
                'cookie': cookie,
                'User-Agent': ua
            }
        });
        }
        catch (error) {
            const e = getError(error);
            if (e.message === 'Response code 404 (Not Found)') {
                throw new Error('Not found');
            }
            throw e;
        }
        return assembleDoujin(response);
    }

Modify this code with your cookies and user agent (must be the user agent used to get the cookies)
The code must be run with the same IP adress than the browser used to get the cookies or use a VPS/VPN.

[trueimmortal]

@jodu75 you can use triple back ticks to insert code in github btw

console.Writeline("like this");

[zYxDevs]

@sleroq maybe this repo will be useful for you
https://github.com/sinkaroid/jandapress

or can you make features scraping my own nhentai api deployed from above repo.

thank you:)