Implement smart-rank-model using updated smart-model patterns

- Updated README.md files for both smart-model and smart-rank-model to improve clarity and structure, including new sections on features, usage, and folder structure.
- Introduced a new .scignore file in smart-rank-model to manage ignored files for the Smart Context VSCode extension https://github.com/brianpetro/smart-context-vscode
- Enhanced adapter management in smart-rank-model by adding new adapters (Cohere, Transformers Worker, and Transformers Iframe) and improving existing ones for better integration and functionality.
- Updated package.json to include the latest version of @huggingface/transformers for improved model support.
- Refactored the SmartRankModel class to streamline model loading and ranking processes, including better error handling and configuration management.
- Improved testing framework for both Cohere and Transformers adapters, ensuring robust validation of ranking functionality across various models.

This update significantly enhances the usability and maintainability of the smart-model and smart-rank-model packages, providing a more cohesive development experience.

smart-model/README.md

@@ -1,41 +1,61 @@
-# Smart Model
+# smart-model
+
+A flexible base class for building "smart" models that handle configuration, state management, adapter loading, and model operations. Designed to provide a standardized interface, it simplifies creating specialized model classes (e.g., language models, ranking models) with different backends and adapters.
 
 Base class for smart-*-model packages.
 
+## Features
+- Base `SmartModel` class manages:
+  - Adapter lifecycle (load/unload)
+  - Settings configuration and schema processing
+  - State transitions (`unloaded`, `loading`, `loaded`, `unloading`)
+- Extensible architecture for multiple adapters
+- Centralized settings management and re-rendering triggers
 
-## Usage
+## Folder Structure
+```
+smart-model
+├── adapters
+│   └── _adapter.js        # Base adapter class
+├── components
+│   └── settings.js        # Helper for rendering settings UIs
+├── smart_model.js          # Core SmartModel class
+├── test
+│   └── smart_model.test.js # Unit tests for SmartModel
+└── package.json
+```
 
-1. Create an instance of `SmartModel` with the required configuration options:
+## Getting Started
 ```javascript
+import { SmartModel } from 'smart-model';
+
 const model = new SmartModel({
-    adapters: { mock: MockAdapter },
-    settings: { model_key: 'mock_model' },
-    model_config: { adapter: 'mock' }
+  adapters: { myAdapter: MyAdapterClass },
+  settings: { model_key: 'my_model' },
+  model_config: { adapter: 'myAdapter' }
 });
+await model.initialize(); // Loads adapter
 ```
 
-2. Initialize the model (loads the specified adapter):
-```javascript
-await model.initialize();
-```
-
-3. Use adapter-specific methods:
+## Extending
+Subclass `SmartModel` to add domain logic:
 ```javascript
-const result = await model.invoke_adapter_method('mock_method', 'test input');
-console.log(result); // "Processed: test input"
+class MyCustomModel extends SmartModel {
+  get default_model_key() { return 'my_model'; }
+  async custom_method() {
+    return await this.invoke_adapter_method('some_adapter_method');
+  }
+}
 ```
 
+## Adapters
+Adapters bridge between `SmartModel` and external APIs or local logic. Create a subclass of `SmartModelAdapter` and implement necessary methods (e.g., `load`, `rank`, `invoke_api_call`).
 
+## Testing
+Run tests:
+```
+npm test
+```
 
-
-## State Transitions
-
-The `SmartModel` instance has the following states:
-- `unloaded`: No adapter is loaded.
-- `loading`: Adapter is in the process of being loaded.
-- `loaded`: Adapter has been successfully loaded.
-- `unloading`: Adapter is in the process of being unloaded.
-
-These states are managed automatically when calling `load` and `unload`.
-
-
+## License
+MIT

smart-rank-model/.scignore

@@ -0,0 +1,4 @@
+# Ignore files when getting folder as context using Smart Context VSCode extension https://github.com/brianpetro/smart-context-vscode
+cl100k_base.json
+connectors/**
+test/*.json

smart-rank-model/README.md

@@ -1,36 +1,77 @@
-# @smart-rank-model
+# smart-rank-model
 
-Convenient interface for utilizing various ranking models via API and locally.
+`smart-rank-model` extends `smart-model` for ranking tasks. It supports multiple adapters (e.g., Cohere, transformers) to reorder documents based on a given query, providing a unified interface for switching backend models seamlessly.
 
 ## Features
+- Extends `SmartModel` to handle ranking tasks.
+- Provides a `rank(query, documents)` method, returning ranked results.
+- Integrates adapters (API-based, local transformers, iframes, workers) to accommodate different runtime environments.
+- Includes a curated `models.json` mapping adapter names to known model configurations.
 
-- Supports multiple ranking models
-- Flexible adapter system for different model implementations
-- GPU acceleration support (when available)
-- Easy-to-use API for document ranking
+## Folder Structure
+```
+smart-rank-model
+├── adapters
+│   ├── cohere.js              # Cohere API adapter
+│   ├── transformers.js        # Local transformers adapter (Hugging Face)
+│   ├── transformers_iframe.js # Transformers via iframe
+│   ├── transformers_worker.js # Transformers via Web Worker
+│   ├── _adapter.js            # Base ranking adapter
+│   ├── _api.js                # Base API adapter class
+│   └── _message.js            # Base message-based adapter class
+├── connectors                 # Scripts used by iframe/worker adapters
+├── build                      # Build scripts for connectors
+├── test
+│   ├── cohere.test.js
+│   ├── transformers.test.js
+│   └── _env.js
+├── models.json                # Known model configurations
+├── smart_rank_model.js        # Core SmartRankModel class
+└── package.json
+```
 
-## Configuration
+## Usage
+```javascript
+import { SmartRankModel } from 'smart-rank-model';
+import { cohere as CohereAdapter } from 'smart-rank-model/adapters.js';
 
-The `SmartRankModel` constructor accepts two parameters:
+const rankModel = new SmartRankModel({
+  model_key: 'cohere-rerank-english-v3.0',
+  adapters: { cohere: CohereAdapter },
+  model_config: { api_key: 'YOUR_COHERE_API_KEY', endpoint: "https://api.cohere.ai/v1/rerank" }
+});
+await rankModel.initialize();
 
-1. `env`: The environment object containing adapter configurations
-2. `opts`: Model configuration options
+const query = "What is the capital of the United States?";
+const documents = [
+  "Carson City is the capital city of Nevada.",
+  "Washington, D.C. is the capital of the United States."
+];
+const results = await rankModel.rank(query, documents);
+console.log(results);
+```
 
-### Model Options
+## Switching Adapters
+Change `model_key` and `adapter` to switch between Cohere, transformers, or other supported backends. `models.json` provides a quick lookup of supported models.
 
-- `model_key`: Identifier for the model in the `models.json` file
-- `adapter`: The adapter to use for this model
-- `use_gpu`: Boolean to enable/disable GPU acceleration (auto-detected if not specified)
-- `gpu_batch_size`: Batch size for GPU processing (default: 10)
+## Local Transformers
+Use `transformers` adapter to run local ranking models via WebAssembly, WebGPU, or CPU fallback:
+```javascript
+import { transformers as TransformersAdapter } from 'smart-rank-model/adapters.js';
 
-## Adapters
+const localModel = new SmartRankModel({
+  model_key: 'jinaai/jina-reranker-v1-tiny-en',
+  adapters: { transformers: TransformersAdapter },
+});
+await localModel.initialize();
+const response = await localModel.rank("organic skincare", ["Some doc", "Another doc"]);
+```
 
-Adapters should be implemented and added to the `env.opts.smart_rank_adapters` object. Each adapter should implement the following methods:
-
-- `constructor(model)`: Initialize the adapter
-- `load()`: Load the model
-- `rank(query, documents)`: Rank the documents based on the query
+## Testing
+Run tests:
+```
+npm test
+```
 
 ## License
-
-MIT License. See `LICENSE` file for details.
+MIT

smart-rank-model/adapters.js

@@ -1,10 +1,13 @@
 import { SmartRankAdapter } from "./adapters/_adapter.js";
+import { SmartRankCohereAdapter } from "./adapters/cohere.js";
 import { SmartRankTransformersAdapter } from "./adapters/transformers.js";
 import { SmartRankTransformersIframeAdapter } from "./adapters/transformers_iframe.js";
+import { SmartRankTransformersWorkerAdapter } from "./adapters/transformers_worker.js";
 
 export {
   SmartRankAdapter as _default,
   SmartRankCohereAdapter as cohere,
   SmartRankTransformersAdapter as transformers,
   SmartRankTransformersIframeAdapter as transformers_iframe,
-};
+  SmartRankTransformersWorkerAdapter as transformers_worker,
+};

smart-rank-model/adapters/_adapter.js

@@ -1,10 +1,33 @@
 import { SmartModelAdapter } from "smart-model/adapters/_adapter.js";
 
-export class SmartRankModelAdapter extends SmartModelAdapter {
+/**
+ * Base adapter class for ranking models
+ * @abstract
+ * @class SmartRankAdapter
+ * @extends SmartModelAdapter
+ */
+export class SmartRankAdapter extends SmartModelAdapter {
+  /**
+   * Create a SmartRankAdapter instance.
+   * @param {SmartRankModel} model - The parent SmartRankModel instance
+   */
   constructor(model) {
     super(model);
+    /**
+     * @deprecated Use this.model instead
+     */
     this.smart_rank = model;
   }
-  async count_tokens(input) { throw new Error("Not implemented"); }
-  async rank(query, documents) { throw new Error("Not implemented"); }
+
+  /**
+   * Rank documents based on a query.
+   * @abstract
+   * @param {string} query - The query string
+   * @param {Array<string>} documents - The documents to rank
+   * @returns {Promise<Array<Object>>} Array of ranking results {index, score, ...}
+   * @throws {Error} If the method is not implemented by subclass
+   */
+  async rank(query, documents) {
+    throw new Error('rank method not implemented');
+  }
 }