How to solve projection combined with populated projection data forwards on MongoDB 4.4?

[EmilsWebbod]

Hello, started on updating from version 4.2 to 4.4 and ran into some issues with projection and populations.
I've added some simple node.js code test that works in 4.2 but not in 4.4.

Tried with
Mongo DB: v4.2 / v4.4
mongoose: v5.11.18

4.4 gives me this error message MongoError: Path collision at items.ref remaining portion ref

Appreciate any help and how have you solved this?

Original request

const user= await User.findOne(
      { _id: user._id, 'profile._id': profileId },
      {  'profile.$': 1 }
    )
      .populate({
        path: 'profile.page profile.organization profile.book profile.chapter profile.part',
        select: 'slug'
      })

Test code localy

const mongoose = require('mongoose');

const CONNECTION_STRING = 'mongodb://localhost:27017/projection-population';
const opts = {
  useNewUrlParser: true,
  useUnifiedTopology: true,
  w: 'majority',
  retryWrites: true,
  useFindAndModify: false
};
mongoose.connect(CONNECTION_STRING, opts, onConnect);

const UserSchema = new mongoose.Schema({
  items: {
    type: [
      {
        data: String,
        ref: { type: mongoose.Types.ObjectId, ref: 'Item' }
      }
    ],
    default: []
  }
});

const ItemSchema = new mongoose.Schema({
  name: { type: String, required: true },
  created: { type: Date, default: Date.now }
});

const User = mongoose.model('User', UserSchema);
const Item = mongoose.model('Item', ItemSchema);

async function onConnect() {
  // Clear DB
  await User.deleteMany({});
  await Item.deleteMany({});

  // Add content
  const items = ['One', 'Two'].map((x) => ({
    _id: mongoose.Types.ObjectId(),
    name: x
  }));
  await Item.create(...items);
  const user = await User.create({
    items: items.map((x) => ({ data: 'SOME DATA', ref: x._id }))
  });

  // Try to find, project matched item then populate ref
  const found = await User.findOne(
    { _id: user._id, 'items.ref': items[1]._id },
    { 'items.$': 1 }
  ).populate({ path: 'items.ref', select: 'name' });

  console.log(found.items[0].name === 'Two'));
}

Solutions

Populate
Simple fix, but should not be necessary.
I see this is probably the way mongoose does this today under the hood. Looking at debug logs I can see that it has its own request after it has found data with help of { _id: { $in: [...] } } for items

const user = await User.find(query, project);
await Item.populate(user, { path: 'items.ref' })

Filter after
Fetching data from DB that's not used. Slower requests.

const user = await User.find(query).populate({ path: 'items.ref' });
user.items.filter(byId(itemID))

Aggregate
Good solution with 1 DB call and only gets necessary data. But adds 10x more code.
Added the steps I think is needed to do this, wont waste time on writing it down here

const user = await User.aggregate(
[$match, $project, $unwind, $match, $lookup x models]
)

Solve in mongoose
I dont like any of those solutions. It will add more code and complexity that shouldn't be necessary. Its so much cleaner on how it was solved in 4.2.

Could there be a change in mongoose that matches some strings so it wont send projections that don't match when user uses
Now find(query, { 'items.$': 1 ).populate('items.ref') = { projection: { 'items.$': 1, 'items.ref': 1 }
Change find(query, { 'items.$': 1 ).populate('items.ref') = { projection: { 'items.$': 1 }

I also see this fails when user select that specific field. This seems like a strange error when its more expressive on what you really are doing.

const user = await User.find(query, { 'items.$': 1 }).select('name items')

[EmilsWebbod]

Bump