importccl: the userfile table name generation algorithm breaks with special usernames, and confused client-side and server-side usernames

[knz]

This code:

 qualifiedTableName = DefaultQualifiedNamePrefix + user

(external_storage.go)

breaks when user starts with digits, or when it contains periods. Username consisting of only digits, or containing periods, have been allowed since crdb v19.2.

The code should instead quote the username properly.

[knz]

Additionally, the userfile CLI commands takes the username in the connection URL, and uses it to assemble a destination file name in the COPY IN statement in SQL.

This is conceptually mistaken, because the client-side username may not be the same as the server-side username.

The code should be modified to extract the server-side username using the session_user session variable.

[knz]

Had some exchange with @dt on this:

@knz the logic creates invalid table names / identifiers given some input usernames. Suppose I was a user with such a special username. Can you enumerate exactly what would break for me? What would I not be able to do?

@dt I think you’d not be able to use userfile without supplying the optional table name argument. We default to a username-based table name but you can specify any table if you want.

@knz what is the lifetime of "userfile" data? Is it transient? Does it last for more than a single SQL statement or job?

@dt Yes, we do not use temp tables or anything with a lifetime tied to a session or anything, since we upload in a separate session and jobs survive eg restarts. Currently only the user chooses when / if to delete userfile data, but 21.1 will include a cockroach import path-to-file.sql which would automate a) calling upload b) sending the IMPORT stmt of the uploaded file c) rm’ing the uploaded file.

@knz the previous question is really a proxy for this one: if I were to change the name generation algorithm, do I need to care about cross-version compatibility?

@dt Yeah, it would be nice if no-name-provided file uploads were still findable at the default name in a future version but if we have to break it for everyone and go to a totally new name format, I wouldn’t be devastated

@knz is the name embedded inside the job payload? so that if we change the gen algo, all current and past jobs are still attached to the right name?

@dt Yeah, gen algo should be nearly entirely at cli -- my mental model is that it is basically the default value for a cli flag but that substitution should happen very early and use the actual value after that

[knz]

Ok so a good next step here would be:

1. factor all the code paths that create a table name from a username inside a helper function (I counted at least 4 occurrences, there may be more?)

2. define a new algorithm that works OK for any username, but does a best effort at preserving compatibility with the current one.

I am thinking about an algorithm that looks like this:

1. compose "userfile_" + username.
2. is the result a valid unquoted SQL identifier and it is normalized already? If yes, then keep it.
3. otherwise, compose "userfilex_" + fmt.Sprintf("%x", username) and use that

This would escape the entire username to hexadecimal if not a lowercase normalized identifier already.

Examples:

Username	Userfile table name
root	userfile_root
ro.ot	userfilex_726f6f74
726f6f74	userfile_726f6f74

Regardless of the algorithm used, the end result must ensure that:

• if 2 usernames are different, their mapping to a userfile table are also different.
• the set of allowable characters in usernames can grow/evolve without requiring changes to the bulk i/o code

The hexification of the username combined with a x_ guarantee this. Other schemes may also be possible.

[adityamaru]

Thanks for this @knz, I haven't been able to get to this yet but I'm clear on the behaviour we expect now. I'll write up a PR and add you to it!