-
Notifications
You must be signed in to change notification settings - Fork 11.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sql_require_primary_key Causes Tables With String Primary Key To Fail #33238
Comments
Hmm this is a tricky one. Apparently the SQL generated to create the table doesn't immediately adds the primary key and thus indeed causes this error. I'm not sure how (or if) this can be solved. |
Thanks for taking a look! I know it doesn't help in finding a resolution, but it appears that this also affects Passport install. See here: https://github.com/laravel/passport/blob/9.x/database/migrations/2016_06_01_000001_create_oauth_auth_codes_table.php
|
In my case I use Digital Ocean as well, but this problem happened for pivot tables only and I had to edit the migrations by adding |
@erwinweber96 the problem is that that won't work for string based primary identifiers. Thinking password resets table, database notifications, passport, etc. |
I agree that this needs to be solved, however I added that comment in case someone stumbles upon the issue (like I did) regarding sql_require_primary_key and laravel. There's very few info online on this matter atm. |
I'm experiencing the same issue. Unsure if it was a recent change to DigitalOcean's config but Laravel applications utilising Passport are effectively incompatible with DigitalOcean Managed Databases at the moment. |
update below solution won't work if you run tests in sqlite for example. See #33238 (comment) for a simpler workaround. Same issue here, it affects the For anyone that needs a fix, this is my workaround for now:
sessions \Illuminate\Support\Facades\DB::statement('CREATE TABLE `sessions` (
`id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`user_id` bigint(20) unsigned DEFAULT NULL,
`ip_address` varchar(45) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`user_agent` text COLLATE utf8mb4_unicode_ci,
`payload` text COLLATE utf8mb4_unicode_ci NOT NULL,
`last_activity` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;'); password_resets \Illuminate\Support\Facades\DB::statement('CREATE TABLE `password_resets` (
`email` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`token` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`token`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;') Laravel TelescopeSee https://laravel.com/docs/7.x/telescope#migration-customization on how to use your own migrations instead of the default ones telescope_entries_tags \Illuminate\Support\Facades\DB::statement('CREATE TABLE `telescope_entries_tags` (
`entry_uuid` char(36) COLLATE utf8mb4_unicode_ci NOT NULL,
`tag` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`entry_uuid`,`tag`),
KEY `telescope_entries_tags_tag_index` (`tag`),
CONSTRAINT `telescope_entries_tags_entry_uuid_foreign` FOREIGN KEY (`entry_uuid`) REFERENCES `telescope_entries` (`uuid`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;'); telescope_monitoring DB::statement('CREATE TABLE `telescope_monitoring` (
`tag` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`tag`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;'); |
Current workaround is setting And I change to
|
Unable to do this on managed database instances (DigitalOcean) as far as I'm aware? |
Yep, this is following a recent change to their config From the tagging of this issue (by Dries as a bug and help wanted), I think there is an acceptance that this needs to be resolved in the framework in some way. Although I acknowledge that is is a fairly sizeable job to sort! |
If someone wants to submit a PR to somehow make this one query behind the scenes be our guest 😄 Currently, this is not a "bug" in Laravel. It is a bug in DigitalOcean's configuration they are forcing upon you. |
I've had the following response from DigitalOcean regarding this issue:
Therefore I managed to get this to work by calling |
Can confirm this works for me as well. Thanks Alex! |
Any latest update or any other solution as of now? The major tables that are affected are the pivot tables only. |
Thank you works for me as well. You can add this into
so it works every time you use the migrate command. |
love you so much |
I'm trying to import an existing Laravel database (MySQL 5.x) into DigitalOcean Managed Database (MySQL 8) and I'm getting that error. How would I set that variable? At the top of the dump file? Thank you! |
Are you using a Tool like TablePlus? Then you can login to the database, execute |
It works, thanks!
And for some Laravel tables like password_resets? Should we add a primary key to a column of choice?
… On 26 Sep 2020, at 08:35, Simon Hansen ***@***.***> wrote:
I've had the following response from DigitalOcean regarding this issue:
I understand you are getting error when trying to migrate database schemas. You can temporarily override the setting using SET SESSION sql_require_primary_key = 1;
However, You will need to add a primary key after you import or you will receive notifications at a point that adding primary key is required.
This setting is only for the current session though, it's not a permanent override.
Therefore I managed to get this to work by calling \Illuminate\Support\Facades\DB::statement('SET SESSION sql_require_primary_key=0'); above my Schema::create command in the migrations causing issues.
I'm trying to import an existing Laravel database (MySQL 5.x) into DigitalOcean Managed Database (MySQL 8) and I'm getting that error. How would I set that variable? At the top of the dump file? Thank you!
Are you using a Tool like TablePlus? Then you can login to the database, execute SET SESSION sql_require_primary_key=0 manually and import your dump. This works because it's the same session :-)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
We didn't make other changes because the error only occurs while creating the table.. after that everything should work as expected. But keep in mind that you cannot create more tables without a primary key (or you have to apply the workaround again). |
Tnx alot!! |
Where in that file should i use it ? |
Yeah where should we add in that file? |
Hi guys! I opened a ticket with DO and they changed the parameter for me. So no need for any "hacky" stuff ;)
|
One simple solution is add this code at // imports
use Illuminate\Support\Facades\Event;
use Illuminate\Database\Events\MigrationsEnded;
use Illuminate\Database\Events\MigrationsStarted;
// code in `register` method
Event::listen(MigrationsStarted::class, function (){
if (env('ALLOW_DISABLED_PK')) {
DB::statement('SET SESSION sql_require_primary_key=0');
}
});
Event::listen(MigrationsEnded::class, function (){
if (env('ALLOW_DISABLED_PK')) {
DB::statement('SET SESSION sql_require_primary_key=1');
}
}); Or you can add a config variable, for example Controlled if you want run |
Only don't use env() in your other files besides your config files. Or ask DO to change the sql requirement for you. |
Same, i use the |
I don't think this issue should be closed because current workarounds are not solving the root cause. |
so you still added the incremental id even if you have the combined primary? and it worked? |
How did you get this to work? |
I would like to say we decided to remove Telescope from the project because of missing primary keys and infinite Digital Ocean warnings.
Sad, but nothing to do if maintainers refuse of primary keys. |
Never ever disable primary keys on HA MySQL and MariaDB clusters as this might lead to corruption of replication between server instances. Probably developers that made this migrations where not aware of how MySQL and MariaDB replication works, especially how Galera, and simply decided as they thought not to add primary keys. |
Should this issue be reopened since the fix was reverted and the issue still exists? |
I do so. Any table MUST have primary keys. And the framework, should test any DB transaction against most limited setup of MariaDB or MySQL, that being default Primary/Replica and Galera setups. Testing against single instance mode does not prove the framework will be trustful in HA setups. Today is very common to deploy on HA setups, and I wouldn't find a reason not to do it so. |
@taylorotwell Would you consider reopening the issue? You say above that it's a "bug" with the Digital Ocean setup but in reality it's a recommended and increasingly common way to configure MySQL and simply turning off required primary keys is not a solution. I believe as things stand several first-party laravel packages such as Nova and Telescope contain migrations that simply can't be run in these setups, and it also makes it a pain to use uuid primary ids. I did look at the revert commit that was merged earlier but couldn't find much context about why the fix was reverted so forgive me if there's a good reason to NOT fix this! |
DO engineers team has updated API documentation to make database configuration changes via API; you will now be able to disable
Then they add:
|
The thing is I don't want to turn off this setting just in order to use uuid primary keys. This is the recommended setting for a reason and i want to retain it. I don't believe it should be necessary to use a less reliable mysql setup because of the way these migrations run under the hood |
@joe-pritchard Then so far the best solution came from @JVillator0
|
Yes that's what I'm doing right now but it's not ideal as it essentially turns off the primary key requirement completely, since all migrations will run without it meaning there's no functional difference to just turning it off in the db's config. Again this is not a solution to the problem, it's a workaround, and I believe there is value in just fixing it at source |
Turning off the requirement is a bit like intentionally incurring technical debt - you know it's going to come back and bite you, just when you're in a rapid scaling situation. |
Note: don't use primary keys and in case of replication failure, deal with the problem and accept also the financial loses for the downtime caused. |
Laravel 8 + DigitalOcean database💡 Just add the first statement to your migration: use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
use Illuminate\Support\Facades\DB;
class AddTableTemplates extends Migration
{
public function up()
{
// fixes DigitalOcean sql_require_primary_key problem
DB::statement('SET SESSION sql_require_primary_key=0');
Schema::create('objects', function (Blueprint $table) {
$table->string('id', 30)
->primary();
// ...
});
}
public function down()
{
Schema::dropIfExists('objects');
}
} |
I encountered this issue myself recently and I think I found a better work around.
It’s pretty similar to those who suggested dumping the schema using another tool such as TablePlus but uses the new On the machine you use for development create a fresh database using (this will delete all existing tables and re-create them using your migrations):
Then dump your schema:
Then commit the changes and push them to the server. Now when you run the migrate command, Laravel will restore the database using the dumped schema file before running the migrations. If you make any changes to the migrations, you will have to re-dump the schema file using this method. Once you are in production though, you shouldn’t have to do that again. Hope this helps; sorry for commenting on an old issue; thanks Taylor and the Laravel Team for your amazing work 😁 |
If anyone else who comes across this thread has a similar issue on Azure. sql_generate_invisible_primary_key <- Make sure this one is turned off as well :D |
Digital Ocean has an API that you can use to configure the settings of your DB, which I would highly recommend over other solutions. You can do the following:
Now your database cluster will be configured, permanently, without any code changes. |
Should be fixed by: #49374 |
Description:
When MySQL has
sql_require_primary_key
enabled, migrations that call->primary()
on a column during table creation fail because it appears as though two queries are ran. One to create the table, and then a second to alter the table and add the primary key.For context, DigitalOcean recently enforced primary keys on all newly created tables in their managed databases.
Steps To Reproduce:
sql_require_primary_key
is enabled on the MySQL server$table->string('string')->primary();
) and does not have a default$table->id();
columnThe below error is generated when attempting to run the first query to create table.
The text was updated successfully, but these errors were encountered: