Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling negative entries in a Case Insensitive file system #4243

Closed
RichardSharpe opened this issue Jan 19, 2016 · 8 comments
Closed

Handling negative entries in a Case Insensitive file system #4243

RichardSharpe opened this issue Jan 19, 2016 · 8 comments
Milestone

Comments

@RichardSharpe
Copy link
Contributor

In an issue that I inadvertently closed, I said:

I have tested both the only returning NULL approach and the approach above.

The test I use is:

1. ls some-file.txt # This should return ENOENT when it does not exist.
2. touch Some-file.txt # Create a file with a name differing only by the case of the first char
3. ls some-file.txt
If the first step created a negative entry in the dcache, step 3 will fail with ENOENT when it should not.

Returning NULL seems to cause ENOENT in step 3, while doing as above (a more complicated approach) seems to work.

I went back and tested the case of returning NULL with the above test, as my previous test was slightly different, and I found that returning NULL does work correctly. It is also the simplest way to deal with this.

I with the addition of the d_iput operation I also get rid of negative entries on delete.

After we resolve the issue of how to set dentry operations I will submit a patch to clean this all up.

@behlendorf
Copy link
Contributor

@RichardSharpe while your chewing on this exact problem let me suggest a small related improvement you could make to help clean this all up. The zfs_remove() and zfs_link function both have real pathname chunks commented out with a #ifdef HAVE_PN_UTILS. This was done in the early days of the project as a simplification to avoid adding pn_alloc() and pn_free().

If you added those two functions you could remove the #ifdef HAVE_PN_UTILS wrappers, use those two new functions in zpl_lookup(), and in the process bring the code a little closer to back in line with Illumos.

@tuxoko
Copy link
Contributor

tuxoko commented Jan 19, 2016

@RichardSharpe
You don't need to add d_op. Just add this to the end of unlink and rmdir when success.
http://lxr.free-electrons.com/source/fs/xfs/xfs_iops.c#L340

@RichardSharpe
Copy link
Contributor Author

Great, thanks ... that is simpler!

@RichardSharpe
Copy link
Contributor Author

@behlendorf

I see what you mean. Let me look at this, perhaps this weekend ... however, I will fix the negative entry issues first.

RichardSharpe added a commit to RichardSharpe/zfs that referenced this issue Jan 24, 2016
For a Case Insensitive file system we must avoid creating negative
entries in the dentry cache. We must also pass the FIGNORECASE into
zfs_lookup so that special files are handled correctly.

We must also prevent negative dentries from being created when files are
unlinked.

Tested by running fsstress from LTP (10 loops, 10 processes, 10,000 ops.)

Also tested with printks (now removed) to ensure that lookups come to
zpl_lookup when negative should not exist.

Tests:
1.   ls Some-file.txt; touch some-file.txt; ls Some-file.txt
  and ensure no errors.

2.   touch Some-file.txt; rm some-file.txt; ls Some-file.txt
  and ensure that the last ls shows log messages showing the lookup
  went all the way to zpl_lookup.

Thanks to tuxoko for helping me get this correct.

Signed-off-by: Richard Sharpe <[email protected]>
@RichardSharpe
Copy link
Contributor Author

I have created a pull request for the fixups ...

I will look at the issue of pn_alloc and pn_free next.

I am thinking I probably want to allocate one piece of memory and then set up the structure correctly.

RichardSharpe added a commit to RichardSharpe/zfs that referenced this issue Jan 24, 2016
For a Case Insensitive file system we must avoid creating negative
entries in the dentry cache. We must also pass the FIGNORECASE into
zfs_lookup so that special files are handled correctly.

We must also prevent negative dentries from being created when files are
unlinked.

Tested by running fsstress from LTP (10 loops, 10 processes, 10,000 ops.)

Also tested with printks (now removed) to ensure that lookups come to
zpl_lookup when negative should not exist.

Tests:
1.   ls Some-file.txt; touch some-file.txt; ls Some-file.txt
  and ensure no errors.

2.   touch Some-file.txt; rm some-file.txt; ls Some-file.txt
  and ensure that the last ls shows log messages showing the lookup
  went all the way to zpl_lookup.

Thanks to tuxoko for helping me get this correct.

Signed-off-by: Richard Sharpe <[email protected]>
@behlendorf
Copy link
Contributor

@RichardSharpe awesome thanks. I took a quick look at the patch and it looks correct to me.

Yes, I think you can just wrap your existing kmem callers with pn_alloc/pn_free wrappers. I'd suggest adding them as static inline functions in include/linux/vfs_compat.h.

As another slight aside you might take a quick look at the related code disabled by the HAVE_DNLC macro. This macro is never defined so this code, which came from illumos, is always disabled since for Linux it makes more sense to do this in the zpl_* functions where we have access to the dentry. We should probably either remove this code entirely or convert it to say a __sunos__ macro.

@RichardSharpe
Copy link
Contributor Author

OK, now that I have seen what Illumnos does there, it does appear to be as easy as you said.

Give me a few days.

@RichardSharpe
Copy link
Contributor Author

Hmmm, Illumnos has this:

void
pn_alloc(struct pathname *pnp)
{
    pn_alloc_sz(pnp, MAXPATHLEN);
}

Which means that the buffer they are allocating pn_path is actually MAXPATHLEN in length.

RichardSharpe added a commit to RichardSharpe/zfs that referenced this issue Jan 26, 2016
For a Case Insensitive file system we must avoid creating negative
entries in the dentry cache. We must also pass the FIGNORECASE into
zfs_lookup so that special files are handled correctly.

We must also prevent negative dentries from being created when files are
unlinked.

Tested by running fsstress from LTP (10 loops, 10 processes, 10,000 ops.)

Also tested with printks (now removed) to ensure that lookups come to
zpl_lookup when negative should not exist.

Tests:
1.   ls Some-file.txt; touch some-file.txt; ls Some-file.txt
  and ensure no errors.

2.   touch Some-file.txt; rm some-file.txt; ls Some-file.txt
  and ensure that the last ls shows log messages showing the lookup
  went all the way to zpl_lookup.

Thanks to tuxoko for helping me get this correct.

Signed-off-by: Richard Sharpe <[email protected]>
kernelOfTruth pushed a commit to kernelOfTruth/zfs that referenced this issue Feb 3, 2016
For a Case Insensitive file system we must avoid creating negative
entries in the dentry cache. We must also pass the FIGNORECASE into
zfs_lookup so that special files are handled correctly.

We must also prevent negative dentries from being created when files are
unlinked.

Tested by running fsstress from LTP (10 loops, 10 processes, 10,000 ops.)

Also tested with printks (now removed) to ensure that lookups come to
zpl_lookup when negative should not exist.

Tests:
1.   ls Some-file.txt; touch some-file.txt; ls Some-file.txt
  and ensure no errors.

2.   touch Some-file.txt; rm some-file.txt; ls Some-file.txt
  and ensure that the last ls shows log messages showing the lookup
  went all the way to zpl_lookup.

Thanks to tuxoko for helping me get this correct.

Signed-off-by: Richard Sharpe <[email protected]>
Signed-off-by: Chunwei Chen <[email protected]>
Signed-off-by: Brian Behlendorf <[email protected]>
Closes openzfs#4243
nedbass pushed a commit that referenced this issue Mar 23, 2016
For a Case Insensitive file system we must avoid creating negative
entries in the dentry cache. We must also pass the FIGNORECASE into
zfs_lookup so that special files are handled correctly.

We must also prevent negative dentries from being created when files are
unlinked.

Tested by running fsstress from LTP (10 loops, 10 processes, 10,000 ops.)

Also tested with printks (now removed) to ensure that lookups come to
zpl_lookup when negative should not exist.

Tests:
1.   ls Some-file.txt; touch some-file.txt; ls Some-file.txt
  and ensure no errors.

2.   touch Some-file.txt; rm some-file.txt; ls Some-file.txt
  and ensure that the last ls shows log messages showing the lookup
  went all the way to zpl_lookup.

Thanks to tuxoko for helping me get this correct.

Signed-off-by: Richard Sharpe <[email protected]>
Signed-off-by: Chunwei Chen <[email protected]>
Signed-off-by: Brian Behlendorf <[email protected]>
Closes #4243
@behlendorf behlendorf added this to the 0.6.5.6 milestone Mar 23, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants