Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: OptionsError: The database options could not be located/loaded. #845

Closed
alancghizzi opened this issue Jan 17, 2024 · 16 comments
Closed

Comments

@alancghizzi
Copy link

Versions

  • Node: v18.17.1
  • OS: MacOS Sonoma 14.2

Reproduction

When executing db:create/drop, it gives the following error:

OptionsError: The database options could not be located/loaded.
    at OptionsError.notFound
Additional Details

DataSource file:

config();

const configService = new ConfigService();

export const dataSourceOptions: DataSourceOptions & SeederOptions = {
    type: 'mysql',
    host: configService.getOrThrow('DB_HOST'),
    port: configService.getOrThrow('DB_PORT'),
    database: configService.getOrThrow('DB_DATABASE'),
    username: configService.getOrThrow('DB_USERNAME'),
    password: configService.getOrThrow('DB_PASSWORD'),
    entities: ['dist/**/*.entity{.ts,.js}'],
    migrations: ['dist/db/migrations/*.js'],
    migrationsTableName: 'migrations',
};

export default new DataSource(dataSourceOptions);

Steps to reproduce

npm run db:create

What is Expected?

To locate the datasource file correctly

What is actually happening?

Its throwing the following error:
OptionsError: The database options could not be located/loaded. at OptionsError.notFound

It runs when i change the following in the "findDataSource" function:

if (info) {
          let fileExports = await locter.load(info);
          if (isPromise(fileExports)) {
              fileExports = await fileExports;
          }
          if (typeorm.InstanceChecker.isDataSource(fileExports)) {
              return fileExports;
          }
          const defaultExport = locter.getModuleExport(fileExports);
          if (isPromise(defaultExport.value)) {
              defaultExport.value = await defaultExport.value;
          }


          // Added .default to the defaultExport.value 
          if (defaultExport && typeorm.InstanceChecker.isDataSource(defaultExport.value.default)) {
              return defaultExport.value.default;
          }


          if (locter.isObject(fileExports)) {
              const keys = Object.keys(fileExports);
              for(let j = 0; j < keys.length; j++){
                  let value = fileExports[keys[j]];
                  if (isPromise(value)) {
                      value = await value;
                  }
                  if (typeorm.InstanceChecker.isDataSource(value)) {
                      return value;
                  }
              }
          }
      }
@tada5hi
Copy link
Owner

tada5hi commented Jan 17, 2024

@alancghizzi Do you have a cjs or esm project? Do you use Typescript for compilation?

@lav-panchal
Copy link

I am having the same issue. I am trying to use typeorm-extension npm package for seed data into the table.
I have all the configurations like creating migration and running migration using typeORM. But now I need to create a seeder file for table.

Please suggest me. how i can archive this?

Details:

Node JS : v18.16.0
typeorm: ^0.3.19
typeorm-extension: ^3.3.0

Code :

import { DataSource, DataSourceOptions } from "typeorm";
import * as dotenv from "dotenv";
import { TokenEntity } from "./entities/token.entity.js";
dotenv.config();

const getConfig = (env: string) => {
    return {
        type: process.env[`DB_DIALECT_${env}`],
        host: process.env[`DB_HOST_${env}`],
        port: Number(process.env[`DB_PORT_${env}`]),
        username: process.env[`DB_USER_${env}`],
        password: process.env[`DB_PASS_${env}`],
        database: process.env[`DB_NAME_${env}`],
        synchronize: false,
        logging: false,
        entities: [
            'src/migrator/entities/*.entity.ts'
        ],
        subscribers: [],
        migrations: [
            'src/migrator/migrations/*.ts'
        ],
        seeds: [
            'src/migrator/seeds/*.ts'
        ],
        migrationsTableName: 'migrations',
    };
};

const env = process.env.NODE_ENV || 'development';
export const AppDataSource = new DataSource(getConfig(env.toUpperCase()) as DataSourceOptions);
export const tokenListRepository = AppDataSource.getRepository(TokenEntity);

Scripts in package.json

"typeorm": "typeorm-ts-node-esm -d ./src/migrator/index.ts",
"orm-migration:run": "npm run typeorm migration:run",
"orm-migration:generate": "npm run typeorm migration:generate",
"seed:run": "ts-node-esm ./node_modules/typeorm-extension/bin/cli.mjs seed:run"

On run : npm run seed:run

ERROR : OptionsError: The database options could not be located/loaded.

Please suggest me with solutions.

@tada5hi
Copy link
Owner

tada5hi commented Jan 17, 2024

@lav-panchal mhhh, from your code snippet, I deduced that you are running an esm project. I tried to replicate your problem.
But without success.
Can you tell me how your code differs from the following code example?
https://github.com/tada5hi/typeorm-extension-test-esm

After you have run npm install, you can test the seeding process with npm run seed.

What is your tsconfig content ?

@alancghizzi
Copy link
Author

@tada5hi the main diff i noticed is that i dont import "reflect-metadata" in my datasource, could that be an issue?

ps: its cjs and yes, i use ts for compilation

This is my datasource file (just replaced the import names to make it clearer):

import { ConfigService } from '@nestjs/config';
import { config } from 'dotenv';
import { DataSource, DataSourceOptions } from 'typeorm';
import { SeederOptions } from 'typeorm-extension';

import Thing1 from './seeding/seeds/thing1.seed';
import Thing2 from './seeding/seeds/thing2.seed';
import Thing3 from './seeding/seeds/thing3.seed';
import Thing4 from './seeding/seeds/thing4.seed';
import Thing5 from './seeding/seeds/thing5.seed';

config();

const configService = new ConfigService();

export const dataSourceOptions: DataSourceOptions & SeederOptions = {
    type: 'mysql',
    host: configService.getOrThrow('DB_HOST'),
    port: configService.getOrThrow('DB_PORT'),
    database: configService.getOrThrow('DB_DATABASE'),
    username: configService.getOrThrow('DB_USERNAME'),
    password: configService.getOrThrow('DB_PASSWORD'),
    entities: ['dist/**/*.entity{.ts,.js}'],
    migrations: ['dist/db/migrations/*.js'],
    migrationsTableName: 'migrations',
    seeds: [Thing1, Thing2, Thing3, Thing4, Thing5],
};

export default new DataSource(dataSourceOptions);

my tsconfig:

{
    "compilerOptions": {
        "module": "commonjs",
        "declaration": true,
        "removeComments": true,
        "emitDecoratorMetadata": true,
        "experimentalDecorators": true,
        "allowSyntheticDefaultImports": true,
        "target": "ES2021",
        "sourceMap": true,
        "outDir": "./dist",
        "baseUrl": "./",
        "incremental": true,
        "skipLibCheck": true,
        "strictNullChecks": false,
        "noImplicitAny": false,
        "strictBindCallApply": false,
        "forceConsistentCasingInFileNames": false,
        "noFallthroughCasesInSwitch": false,
        "paths": {
            "src/*": [
                "./src/*"
            ]
        }
    }
}

@lav-panchal
Copy link

@lav-panchal mhhh, from your code snippet, I deduced that you are running an esm project. I tried to replicate your problem. But without success. Can you tell me how your code differs from the following code example? https://github.com/tada5hi/typeorm-extension-test-esm

After you have run npm install, you can test the seeding process with npm run seed.

What is your tsconfig content ?

tsconfig.json

{
  "compilerOptions": {
    "target": "ESNext",
    "module": "ESNext",
    "moduleResolution": "Node",
    "lib": [
      "esnext"
    ],
    "allowJs": true,
    "esModuleInterop": true,
    "allowSyntheticDefaultImports": true,
    "sourceMap": true,
    "outDir": "build",
    "baseUrl": "src",
    "skipLibCheck": true,
    "strict": true,
    "resolveJsonModule": true,
    "experimentalDecorators": true,
    "strictPropertyInitialization": false,
    "emitDecoratorMetadata": true,
  },
  "include": [
    "src/**/*.ts",
    "tests/**/*.ts",
  ],
  "exclude": [
    "node_modules"
  ]
}

@lav-panchal
Copy link

@tada5hi Now getting this error:

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /home/bacancy/Desktop/Projects/backend/src/migrator/index.ts
require() of ES modules is not supported.
require() of home/bacancy/Desktop/Projects/backend/src/migrator/index.ts from /home/bacancy/Desktop/Projects/backend/node_modules/locter/dist/index.mjs is an ES module file as it is a .ts file whose nearest parent package.json contains "type": "module" which defines all .ts files in that package scope as ES modules.
Instead change the requiring code to use import(), or remove "type": "module" from /home/bacancy/Desktop/Projects/backend/package.json.

    at createErrRequireEsm (/home/project/.nvm/versions/node/v18.16.0/lib/node_modules/ts-node/dist-raw/node-internal-errors.js:46:15)
    at assertScriptCanLoadAsCJSImpl (/home/project/.nvm/versions/node/v18.16.0/lib/node_modules/ts-node/dist-raw/node-internal-modules-cjs-loader.js:584:11)
    at Object.require.extensions.<computed> [as .ts] (/home/project/.nvm/versions/node/v18.16.0/lib/node_modules/ts-node/src/index.ts:1610:5)
    at Module.load (node:internal/modules/cjs/loader:1117:32)
    at Function.Module._load (node:internal/modules/cjs/loader:958:12)
    at Module.require (node:internal/modules/cjs/loader:1141:19)
    at require (node:internal/modules/cjs/helpers:110:18)
    at ModuleLoader.loadSync (file:///home/project/Desktop/Projects/backend/node_modules/locter/src/loader/built-in/module/module.ts:157:13)
    at ModuleLoader.execute (file:///home/project/Desktop/Projects/backend/node_modules/locter/dist/index.mjs:344:31)
    at processTicksAndRejections (node:internal/process/task_queues:95:5) {
  code: 'ERR_REQUIRE_ESM',
  expose: undefined,
  logMessage: undefined,
  logLevel: undefined,
  data: undefined,
  [cause]: undefined
}

@tada5hi
Copy link
Owner

tada5hi commented Jan 18, 2024

where is your data source file located ?

@lav-panchal
Copy link

where is your data source file located ?

./src/migrator/index.ts

@tada5hi
Copy link
Owner

tada5hi commented Jan 18, 2024

@tada5hi Now getting this error:

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /home/bacancy/Desktop/Projects/backend/src/migrator/index.ts
require() of ES modules is not supported.
require() of home/bacancy/Desktop/Projects/backend/src/migrator/index.ts from /home/bacancy/Desktop/Projects/backend/node_modules/locter/dist/index.mjs is an ES module file as it is a .ts file whose nearest parent package.json contains "type": "module" which defines all .ts files in that package scope as ES modules.
Instead change the requiring code to use import(), or remove "type": "module" from /home/bacancy/Desktop/Projects/backend/package.json.

    at createErrRequireEsm (/home/project/.nvm/versions/node/v18.16.0/lib/node_modules/ts-node/dist-raw/node-internal-errors.js:46:15)
    at assertScriptCanLoadAsCJSImpl (/home/project/.nvm/versions/node/v18.16.0/lib/node_modules/ts-node/dist-raw/node-internal-modules-cjs-loader.js:584:11)
    at Object.require.extensions.<computed> [as .ts] (/home/project/.nvm/versions/node/v18.16.0/lib/node_modules/ts-node/src/index.ts:1610:5)
    at Module.load (node:internal/modules/cjs/loader:1117:32)
    at Function.Module._load (node:internal/modules/cjs/loader:958:12)
    at Module.require (node:internal/modules/cjs/loader:1141:19)
    at require (node:internal/modules/cjs/helpers:110:18)
    at ModuleLoader.loadSync (file:///home/project/Desktop/Projects/backend/node_modules/locter/src/loader/built-in/module/module.ts:157:13)
    at ModuleLoader.execute (file:///home/project/Desktop/Projects/backend/node_modules/locter/dist/index.mjs:344:31)
    at processTicksAndRejections (node:internal/process/task_queues:95:5) {
  code: 'ERR_REQUIRE_ESM',
  expose: undefined,
  logMessage: undefined,
  logLevel: undefined,
  data: undefined,
  [cause]: undefined
}

what have you changed ?

@lav-panchal
Copy link

what have you changed ?

script : ts-node-esm ./node_modules/typeorm-extension/bin/cli.mjs seed:run -d src/migrator/index.ts

@tada5hi
Copy link
Owner

tada5hi commented Jan 18, 2024

can you please check version 3.4.0-beta.1?

@MINDoSOFT
Copy link

MINDoSOFT commented Jan 18, 2024

Hi @tada5hi , I haven't tested version 3.4.0-beta.1 but I want to add some information here, which might help.

We faced a similar issue recently when running typeorm-extension cli commands and trying to provide the datasource.

So we were getting a weird error that 'TypeORMError: No connection options were found in any orm configuration files.' and this error was happening with typeorm-extension.

After much investigation we found out that typeorm can see the datasource when we do:

export default dataSource;

And typeorm-extension can see the datasource when we do:

module.exports = dataSource;

So we ended up providing one datasource for typeorm and another one for typeorm-extension, because funny thing is that typeorm doesn't work with module.exports, it throws :

Error: Given data source file must contain export of a DataSource instance
    at Function.loadDataSource (/node_modules/typeorm/commands/CommandUtils.js:39:19)
    at async Object.handler (/node_modules/typeorm/commands/MigrationRunCommand.js:41:26)

I hope this helps the situation.

We suspect that it has to do with how the dataSource is nested when we use es module, and findDataSource needs to adjust a little bit to handle this case. When we use commonjs module there is no wrapper object around it so it can read the dataSource properly.

@tada5hi
Copy link
Owner

tada5hi commented Jan 18, 2024

Hi @tada5hi , I haven't tested version 3.4.0-beta.1 but I want to add some information here, which might help.

We faced a similar issue recently when running typeorm-extension cli commands and trying to provide the datasource.

So we were getting a weird error that 'TypeORMError: No connection options were found in any orm configuration files.' and this error was happening with typeorm-extension.

After much investigation we found out that typeorm can see the datasource when we do:


export default dataSource;

And typeorm-extension can see the datasource when we do:


module.exports = dataSource;

So we ended up providing one datasource for typeorm and another one for typeorm-extension, because funny thing is that typeorm doesn't work with module.exports, it throws :


Error: Given data source file must contain export of a DataSource instance

    at Function.loadDataSource (/node_modules/typeorm/commands/CommandUtils.js:39:19)

    at async Object.handler (/node_modules/typeorm/commands/MigrationRunCommand.js:41:26)

I hope this helps the situation.

We suspect that it has to do with how the dataSource is nested when we use es module, and findDataSource needs to adjust a little bit to handle this case. When we use commonjs module there is no wrapper object around it so it can read the dataSource properly.

It would be great if you could have a look at 3.4.0-beta.1 and check if the problem persists.
In theory typeorm-extension should handle esm and cjs as well.
You have an esm project right ? Do you execute the compiled js file or do you use ts-node ?

@MINDoSOFT
Copy link

MINDoSOFT commented Jan 18, 2024

Ahhh I checked and it seems the problem persists with 3.4.0-beta.1, at least the one I have.

The project that I have is commonjs.

And when I run my script (I use the compiled js file) I get this error:

OptionsError: The database options could not be located/loaded.
    at Function.notFound (/Users/s.stamatis/Work/fxpro/middleware/utilities/db-migrations/node_modules/typeorm-extension/dist/index.cjs:37:16)
    at Object.buildDataSourceOptions (/Users/s.stamatis/Work/fxpro/middleware/utilities/db-migrations/node_modules/typeorm-extension/dist/index.cjs:1317:24)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Object.handler (/Users/s.stamatis/Work/fxpro/middleware/utilities/db-migrations/node_modules/typeorm-extension/bin/cli.cjs:119:35)

Also please note that in order to overcome this issue, I went and manually adjusted the package.json of typeorm-extension specifically the exports section and added this entry also :

      "./bin/*": "./bin/*"

@tada5hi
Copy link
Owner

tada5hi commented Jan 18, 2024

Issue should be solved with v3.4.0.

testing-library/user-event#813
https://stackoverflow.com/questions/62717394/export-default-class-exports-double-nested-default

@alancghizzi
Copy link
Author

alancghizzi commented Jan 18, 2024

I can confirm that v3.4.0 has fixed it for me.

@tada5hi tada5hi closed this as completed Jan 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants