Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Data too long for column 'include_sources_subscriber' #692

Closed
jth56000 opened this issue Jul 16, 2019 · 38 comments
Closed

Data too long for column 'include_sources_subscriber' #692

jth56000 opened this issue Jul 16, 2019 · 38 comments
Labels
question

Comments

@jth56000
Copy link

Version

6.2.22

Installation method

RPM package sympa-6.2.22-1.20171002.RHEL7.x86_64

Expected behavior

One list based on a sql query can get subscribers.

Actual behavior

The list cannot get its subscribers. The error in the log is :
Jul 16 16:26:06 tcgesympa2 wwsympa[44060]: err main::#1639 > main::do_edit_list#10765 > Sympa::List::on_the_fly_sync_include#7238 > Sympa::List::sync_include#7134 > Sympa::List::add_list_member#3955 > Sympa::Database::do_prepared_query#382 Unable to execute SQL statement "INSERT INTO subscriber_table (user_subscriber, comment_subscriber, list_subsc riber, robot_subscriber, date_subscriber, update_subscriber, reception_subscriber, topics_subscriber, visibility_subscriber, subscribed_subscriber, included_subscriber, includ e_sources_subscriber, custom_attribute_subscriber, suspend_subscriber, suspend_start_date_subscriber, suspend_end_date_subscriber, number_messages_subscriber) VALUES (?, ?, ?, ?, FROM_UNIXTIME(1563287166), FROM_UNIXTIME(1563287166), ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0)": (22001) Data too long for column 'include_sources_subscriber' at row 1

Jul 16 16:26:06 tcgesympa2 wwsympa[44060]: err main::#1639 > main::do_edit_list#10765 > Sympa::List::on_the_fly_sync_include#7238 > Sympa::List::sync_include#7134 > Sympa::List::add_list_member#3994 Unable to add subscriber [email protected] to table subscriber_table for list [email protected]

Additional information

NONE
@ikedas
Copy link
Member

ikedas commented Jul 17, 2019

This problem may be solved by #693.

As a workaround, you may extend the size of include_sources_subscriber field, such as:

ALTER TABLE subscriber_table CHANGE COLUMN
include_sources_subscriber include_sources_subscriber varchar(62);

(You look using MySQL/MariaDB)

@jth56000
Copy link
Author

Hello Ikeda,

We have modified subscriber_table structure to set include_source_subscriber to varchar(100).
Bu we still have an error in sympa log :

Jul 17 14:50:56 tcgesympa2 wwsympa[46724]: err main::#1639 > main::do_edit_list#10765 > Sympa::List::on_the_fly_sync_include#7238 > Sympa::List::sync_include#7134 > Sympa::List::add_list_member#3955 > Sympa::Database::do_prepared_query#382 Unable to execute SQL statement "INSERT INTO (user_subscriber, comment_subscriber, list_subscriber, robot_subscriber, date_subscriber, update_subscriber, reception_subscriber, topics_subscriber, visibility_subscriber, subscribed_subscriber, included_subscriber, include_sources_subscriber, custom_attribute_subscriber, suspend_subscriber, suspend_start_date_subscriber, suspend_end_date_subscriber, number_messages_subscriber) VALUES (?, ?, ?, ?, FROM_UNIXTIME(1563367856), FROM_UNIXTIME(1563367856), ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0)": (22001) Data too long for column 'include_sources_subscriber' at row 1

And in mariadb log :

INSERT INTO subscriber_table (user_subscriber, comment_subscriber, list_subscriber, robot_subscriber, date_subscriber, update_subscriber, reception_subscriber, topics_subscriber, visibility_subscriber, subscribed_subscriber, included_subscriber, include_sources_subscriber, custom_attribute_subscriber, suspend_subscriber, suspend_start_date_subscriber, suspend_end_date_subscriber, number_messages_subscriber) VALUES ('[email protected]', NULL, 'proriorecette', 'listes-rec.dsi.cnrs.fr', FROM_UNIXTIME(1563368107), FROM_UNIXTIME(1563368107), 'mail', NULL, 'noconceal', '0', '1', '0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26,0e8f8b26', NULL, NULL, NULL, NULL, 0)

Thank you for your help.
Jean

@jth56000
Copy link
Author

I can add this is critical for us, we have alsohave this error when migrating from
RedHat 5.8, sympa 6.2.22 code source install, mysql-server-5.5.25a-1.el5.remi
To
RedHat 7.6, sympa 6.2.22 RPM packages, rh-mariadb102-mariadb-server-10.2.22-1.el7.x86_64

Regards,
Jean

@ikedas
Copy link
Member

ikedas commented Jul 18, 2019

Though I don't know why, inclusion from data source doesn't work properly. A way to fix quickly is as following:

  1. Remove all subscribers who are included from data source(s).
  2. Remove all entries of the subscribers above in "exclusion" table of the list (that is shown by "Exclusion" button).
  3. Synchronize the list with data source(s) again (by waiting for an hour, or synchronize immediately by clicking "Synchronize members with data sources" button).

@jth56000
Copy link
Author

jth56000 commented Jul 18, 2019 via email

@jth56000
Copy link
Author

Hello Ikeda, for your information thi issue is our priority since we cannot migrate our production server.

@ikedas
Copy link
Member

ikedas commented Jul 22, 2019

Hi @jth56000,
Could you please show sql log too?

@jth56000
Copy link
Author

jth56000 commented Jul 22, 2019 via email

@ikedas
Copy link
Member

ikedas commented Jul 22, 2019 •
edited
Loading

This sql log is not releated to sympa log. Please delete it.

@jth56000
Copy link
Author

jth56000 commented Jul 22, 2019 via email

@ikedas
Copy link
Member

ikedas commented Jul 22, 2019

As a workaround, you can add the user by doing:

  1. Remove the subscriber, if they have been included in the list.
  2. Add the subscriber manually, using "Add" button on "Manage list members" page.
  3. Check if the entry of the subscriber above is included in "exclusion" table, or add the entry (the table is shown by "Exclusion" button).
  4. Synchronize the list with data source(s) again (by waiting for an hour, or synchronize immediately by clicking "Synchronize members with data sources" button).
  5. Repeat 1. to 4. by each subscriber that causes error.

@jth56000
Copy link
Author

jth56000 commented Jul 22, 2019 via email

@racke
Copy link
Contributor

racke commented Jul 22, 2019

This sql log is not releated to sympa log. Please delete it.

It could be relevant as the actual value of include_sources_subscriber is shown here, e.g.:

'093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c,093cb65c'

That might be the problem ... why Sympa does insert the same value multiple times?

@ikedas
Copy link
Member

ikedas commented Jul 22, 2019 •
edited
Loading

I can’t do that on the production server. Many lists and uses are impacted by this issue.

First, have you tried the procedure I wrote at 18th July?

If you have done that procedure and error about "Data too long" occurred again, there may be some problem (such as loop on inclusion). So I suggested a workaround.

Second, why can't you try the workaround I wrote on just above?

@jth56000
Copy link
Author

jth56000 commented Jul 22, 2019 via email

@jth56000
Copy link
Author

jth56000 commented Jul 22, 2019 via email

@ikedas
Copy link
Member

ikedas commented Jul 22, 2019

I did the procedure on 18th June but still have the error.

If you have done that procedure and error about "Data too long" occurred again, there may be some problem (such as loop on inclusion). I think that, to investigate the reason of this problem drastically, you once will have to remove all setting about inclusion. It is not realistic idea.

So I suggested a workaround.

@racke
Copy link
Contributor

racke commented Jul 22, 2019

Ok but why does Sympa write these duplicated values?

@jth56000
Copy link
Author

jth56000 commented Jul 22, 2019 via email

@racke
Copy link
Contributor

racke commented Jul 22, 2019

You can safely set it to varchar(255).

@ikedas
Copy link
Member

ikedas commented Jul 22, 2019

The work around you suggested was to set include_sources_subscriber varchar(62). I set it to varchar(100) and still have the issue?

The workaround I suggested is:

As a workaround, you can add the user by doing:

  1. Remove the subscriber, if they have been included in the list.
  2. Add the subscriber manually, using "Add" button on "Manage list members" page.
  3. Check if the entry of the subscriber above is included in "exclusion" table, or add the entry (the table is shown by "Exclusion" button).
  4. Synchronize the list with data source(s) again (by waiting for an hour, or synchronize immediately by clicking "Synchronize members with data sources" button).
  5. Repeat 1. to 4. by each subscriber that causes error.

What is the reason you cannot do it? (it is pure question, not rhetorical.)

@jth56000
Copy link
Author

jth56000 commented Jul 22, 2019 via email

@jth56000
Copy link
Author

jth56000 commented Jul 22, 2019 via email

@ikedas
Copy link
Member

ikedas commented Jul 22, 2019 •
edited
Loading

I see. Now you are working on old server. Is it true?

If it is true, could you please tell us:

  • Is the error occurring on the old server?
  • Version and installation method of the "old" server.

@jth56000
Copy link
Author

jth56000 commented Jul 22, 2019 via email

@ikedas
Copy link
Member

ikedas commented Jul 22, 2019

It means that there are some differences between old and new servers (and environment like network). Please check them at first.

You would be better to try migrating one or a few lists at first, and to see if they will be migrated successful.

@jth56000
Copy link
Author

jth56000 commented Jul 22, 2019 via email

@jth56000
Copy link
Author

jth56000 commented Jul 22, 2019 via email

@ikedas
Copy link
Member

ikedas commented Jul 22, 2019

It’s obvious that there are any difference, because one failed and another didn’t fail.

You may create a test list on the old server and copy it’s configuration to the new server. These lists may not affect existing lists and subscribers.

——
Sorry but I’m living in Japan and will go asleep soon.

@jth56000
Copy link
Author

jth56000 commented Jul 22, 2019 via email

@jth56000
Copy link
Author

jth56000 commented Jul 22, 2019 via email

@jth56000
Copy link
Author

jth56000 commented Jul 23, 2019 via email

@ikedas
Copy link
Member

ikedas commented Jul 28, 2019

This problem may be solved by #693. If you can wait for the next release, you may check if the problem will be solved with current beta, 6.2.45b.2.

@ikedas
Copy link
Member

ikedas commented Aug 22, 2019

@jth56000, could you check if the problem was solved?

@jth56000
Copy link
Author

jth56000 commented Aug 22, 2019 via email

@ikedas
Copy link
Member

ikedas commented Aug 22, 2019

6.2.45b.2 will no longer use include_sources_subscriber field. Did you check this version?

@jth56000
Copy link
Author

jth56000 commented Aug 22, 2019 via email

@ikedas
Copy link
Member

ikedas commented Aug 23, 2019

I undertand. You don't use 6.2.45b.2. If you have no more questions, I'll close this issue.

@ikedas ikedas closed this as completed Aug 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@racke @ikedas @jth56000