Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error: argument for 's' must be a bytes object #46

Open
vwake7 opened this issue Jun 17, 2024 · 1 comment
Open

error: argument for 's' must be a bytes object #46

vwake7 opened this issue Jun 17, 2024 · 1 comment

Comments

@vwake7
Copy link

vwake7 commented Jun 17, 2024

@altaurog Can you please help me check if I am missing something obvious. Any help is appreciated.

I had 16 columns in tables where I am copying to and most of column type is text except two and I get the below error when I have atleast 9 values defined as string. But if I have 8 values as string I am able to execute writestream function without any problem. Down the line I will have issues as I gave only 8 values, but atleast I can go beyond this function. If I gave 16 integers or None this writestream function is executing fine.

error                                     Traceback (most recent call last)
[<ipython-input-47-f54eec967a67>](https://localhost:8080/#) in <cell line: 173>()
    173 while True:
    174   get_data_from_skuspotpricehistory()
--> 175   insert_into_table()
    176   if skuspotpricehistory[0]['$skipToken'] == 'null':
    177     break

2 frames
[<ipython-input-47-f54eec967a67>](https://localhost:8080/#) in insert_into_table()
    128   records = [('None','None','None','None','None','None','None','None','None'),]
    129   #records = [(1,2,3,4,5,6,7,8,9,10,11),]
--> 130   mgr.copy(records)
    131 
    132     # query = """

[/usr/local/lib/python3.10/dist-packages/pgcopy/copy.py](https://localhost:8080/#) in copy(self, data, fobject_factory)
    313         """
    314         datastream = fobject_factory()
--> 315         self.writestream(data, datastream)
    316         datastream.seek(0)
    317         self.copystream(datastream)

[/usr/local/lib/python3.10/dist-packages/pgcopy/copy.py](https://localhost:8080/#) in writestream(self, data, datastream)
    344                 f, d = formatter(val)
    345                 fmt.append(f)
--> 346                 rdat.extend(d)
    347             datastream.write(struct.pack("".join(fmt), *rdat))
    348         datastream.write(BINCOPY_TRAILER)

error: argument for 's' must be a bytes object
@altaurog
Copy link
Owner

It’s hard to diagnose without seeing the table definition and the code. Something is funny with the traceback, since the error is coming from the call to struct.pack, not rdat.extend (in the release version, the failing call is on line 346, but not in your traceback)

None will work for any nullable column. I don’t believe there are any column types for which you can pass either an int or a str. If you are using a recent version of pgcopy, encoding is handled automatically for most text types. The exceptions are bytea and jsonb, for which you must encode the value to a bytes object first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants