Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modularize Aws::Record #149

Merged
merged 8 commits into from
Nov 1, 2024
Merged

Modularize Aws::Record #149

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
c7c33e8
Modularize Aws::Record
mullermp Oct 24, 2024
af0eb2f
Merge branch 'main' into aws-record-refactor
mullermp Oct 28, 2024
2b671b9
Clean up
mullermp Oct 28, 2024
c1bba4d
Fix rubocop
mullermp Oct 28, 2024
a0fdf0b
Merge branch 'main' into aws-record-refactor
mullermp Oct 29, 2024
9779f4c
Merge branch 'main' into aws-record-refactor
mullermp Oct 29, 2024
a1edfa6
assert mock for test table config
mullermp Nov 1, 2024
832d794
Rubocop
mullermp Nov 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .rubocop.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ AllCops:
- 'test/dummy/**/*'
- 'spec/fixtures/**/*'
- 'spec/fixtures/**/*'
- 'db/**/*'

Gemspec/DevelopmentDependencies:
EnforcedStyle: Gemfile
Expand All @@ -34,7 +33,11 @@ Metrics/BlockLength:
Exclude:
- 'spec/**/*.rb'
- 'test/**/*.rb'
- aws-sdk-rails.gemspec

Metrics/ClassLength:
Exclude:
- 'spec/**/*.rb'
- 'test/**/*.rb'

Metrics/ModuleLength:
Exclude:
Expand All @@ -46,7 +49,6 @@ Style/HashSyntax:

Style/Documentation:
Exclude:
- 'lib/generators/**/*.rb'
- 'lib/aws/rails/notifications.rb'
- 'spec/**/*.rb'
- 'test/**/*.rb'
8 changes: 0 additions & 8 deletions .rubocop_todo.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,31 +67,23 @@ Naming/AccessorMethodName:
Naming/PredicateName:
Exclude:
- 'spec/**/*'
- 'lib/generators/aws_record/base.rb'

# Offense count: 9
# Configuration parameters: AllowedConstants.
Style/Documentation:
Exclude:
- 'spec/**/*'
- 'test/**/*'
- 'lib/active_job/queue_adapters/sqs_adapter.rb'
- 'lib/aws/rails/notifications.rb'
- 'lib/aws/rails/sqs_active_job/configuration.rb'
- 'lib/aws/rails/sqs_active_job/job_runner.rb'
- 'lib/aws/rails/sqs_active_job/lambda_handler.rb'
- 'lib/generators/aws_record/base.rb'
- 'lib/generators/aws_record/generated_attribute.rb'
- 'lib/generators/aws_record/model/model_generator.rb'
- 'lib/generators/aws_record/secondary_index.rb'

# Offense count: 5
# This cop supports safe autocorrection (--autocorrect).
Style/IfUnlessModifier:
Exclude:
- 'lib/aws/rails/middleware/ebs_sqs_active_job_middleware.rb'
- 'lib/generators/aws_record/base.rb'
- 'lib/generators/aws_record/secondary_index.rb'

# Offense count: 23
# This cop supports safe autocorrection (--autocorrect).
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
Unreleased Changes
------------------

* Feature - Prepare modularization of `aws-record`.

* Feature - Add session store config generation with `rails generate dynamo_db:session_store_config`. Config generation is no longer tied to the DynamoDB SessionStore ActiveRecord migration generator.

* Feature - Prepare modularization of `aws-sessionstore-dynamodb`.
Expand Down
125 changes: 74 additions & 51 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ This gem also brings in the following AWS gems:
* `aws-sdk-sesv2`
* `aws-sdk-sqs`
* `aws-sdk-sns`
* `aws-record`

You will have to ensure that you provide credentials for the SDK to use. See the
latest [AWS SDK for Ruby Docs](https://docs.aws.amazon.com/sdk-for-ruby/v3/api/index.html#Configuration)
Expand Down Expand Up @@ -628,17 +627,27 @@ FIFO queues require a message group id to be provided for the job. It is determi
2. If `message_group_id` is not defined or the result is `nil`, the default value will be used.
You can optionally specify a custom value in your config as the default that will be used by all jobs.

## AWS Record Generators
## AWS Record Model Generators

This package also pulls in the [\`aws-record\` gem](https://github.com/aws/aws-sdk-ruby-record)
and provides generators for creating models and a rake task for performing
table config migrations.
This package provides generators for creating `Aws::Record` models and table
configurations that are backed by DynamoDB, and a rake task for performing
the migrations.

To enable this feature, add the following to your Gemfile:

```ruby
gem 'aws-record', '~> 2'
```

### Setup

You can either invoke the generator by calling `rails g aws_record:model ...`
You can either invoke the generator by calling

```bash
rails generate aws_record:model ...
```

If DynamoDB will be the only datastore you plan on using you can also set `aws-record-generator` to be your project's default orm with
If DynamoDB will be the only datastore you plan on using you can also set `Aws::Record` to be your project's default orm with:

```ruby
config.generators do |g|
Expand All @@ -647,11 +656,15 @@ end
```
Which will cause `aws_record:model` to be invoked by the Rails model generator.


### Generating a model

Generating a model can be as simple as: `rails g aws_record:model Forum --table-config primary:10-5`
`aws-record-generator` will automatically create a `uuid:hash_key` field for you, and a table config with the provided r/w units
Generating a model can be as simple as:

```bash
rails generate aws_record:model Forum --table-config primary:10-5
```

The generator will automatically create a `uuid:hash_key` field for you, and a table config with the provided r/w units:

```ruby
# app/models/forum.rb
Expand Down Expand Up @@ -682,7 +695,9 @@ end

More complex models can be created by adding more fields to the model as well as other options:

`rails g aws_record Forum post_id:rkey author_username post_title post_body tags:sset:default_value{Set.new}`
```bash
rails generate aws_record Forum post_id:rkey author_username post_title post_body tags:sset:default_value{Set.new}
```

```ruby
# app/models/forum.rb
Expand All @@ -706,7 +721,9 @@ end

Finally you can attach a variety of options to your fields, and even `ActiveModel` validations to the models:

`rails g aws_record:model Forum forum_uuid:hkey post_id:rkey author_username post_title post_body tags:sset:default_value{Set.new} created_at:datetime:db_attr_name{PostCreatedAtTime} moderation:boolean:default_value{false} --table-config=primary:5-2 AuthorIndex:12-14 --required=post_title --length-validations=post_body:50-1000 --gsi=AuthorIndex:hkey{author_username}`
```bash
rails generate aws_record:model Forum forum_uuid:hkey post_id:rkey author_username post_title post_body tags:sset:default_value{Set.new} created_at:datetime:db_attr_name{PostCreatedAtTime} moderation:boolean:default_value{false} --table-config=primary:5-2 AuthorIndex:12-14 --required=post_title --length-validations=post_body:50-1000 --gsi=AuthorIndex:hkey{author_username}
```

Which results in the following files being generated:

Expand All @@ -726,7 +743,7 @@ class Forum
string_attr :post_title
string_attr :post_body
string_set_attr :tags, default_value: Set.new
datetime_attr :created_at, database_attribute_name: "PostCreatedAtTime"
datetime_attr :created_at, database_attribute_name: 'PostCreatedAtTime'
boolean_attr :moderation, default_value: false

global_secondary_index(
Expand All @@ -744,56 +761,62 @@ end
# ...
```

To migrate your new models and begin using them you can run the provided rake task: `rails aws_record:migrate`
### Running Table Config Migrations

The included rake task `aws_record:migrate` will run all of the migrations in
`app/db/table_config`:

### Docs
```bash
rake aws_record:migrate
```

### Model generator attributes

The syntax for creating an aws-record model follows:

`rails generate aws_record:model NAME [field[:type][:opts]...] [options]`
```bash
rails generate aws_record:model NAME [field[:type][:opts]...] [options]
```

The possible field types are:

Field Name | aws-record attribute type
---------------- | -------------
`bool \| boolean` | :boolean_attr
`date` | :date_attr
`datetime` | :datetime_attr
`float` | :float_attr
`int \| integer` | :integer_attr
`list` | :list_attr
`map` | :map_attr
`num_set \| numeric_set \| nset` | :numeric_set_attr
`string_set \| s_set \| sset` | :string_set_attr
`string` | :string_attr

| Field Name | aws-record attribute type |
|----------------------------------|---------------------------|
| `bool \| boolean` | :boolean_attr |
| `date` | :date_attr |
| `datetime` | :datetime_attr |
| `float` | :float_attr |
| `int \| integer` | :integer_attr |
| `list` | :list_attr |
| `map` | :map_attr |
| `num_set \| numeric_set \| nset` | :numeric_set_attr |
| `string_set \| s_set \| sset` | :string_set_attr |
| `string` | :string_attr |

If a type is not provided, it will assume the field is of type `:string_attr`.

Additionally a number of options may be attached as a comma separated list to the field:

Field Option Name | aws-record option
---------------- | -------------
`hkey` | marks an attribute as a hash_key
`rkey` | marks an attribute as a range_key
`persist_nil` | will persist nil values in a attribute
`db_attr_name{NAME}` | sets a secondary name for an attribute, these must be unique across attribute names
`ddb_type{S\|N\|B\|BOOL\|SS\|NS\|BS\|M\|L}` | sets the dynamo_db_type for an attribute
`default_value{Object}` | sets the default value for an attribute
| Field Option Name | aws-record option |
|---------------------------------------------|-------------------------------------------------------------------------------------|
| `hkey` | marks an attribute as a hash_key |
| `rkey` | marks an attribute as a range_key |
| `persist_nil` | will persist nil values in a attribute |
| `db_attr_name{NAME}` | sets a secondary name for an attribute, these must be unique across attribute names |
| `ddb_type{S\|N\|B\|BOOL\|SS\|NS\|BS\|M\|L}` | sets the dynamo_db_type for an attribute |
| `default_value{Object}` | sets the default value for an attribute |

The standard rules apply for using options in a model. Additional reading can be found [here](#links-of-interest)

Command Option Names | Purpose
-------------------- | -----------
[--skip-namespace], [--no-skip-namespace] | Skip namespace (affects only isolated applications)
[--disable-mutation-tracking], [--no-disable-mutation-tracking] | Disables dirty tracking
[--timestamps], [--no-timestamps] | Adds created, updated timestamps to the model
--table-config=primary:R-W [SecondaryIndex1:R-W]... | Declares the r/w units for the model as well as any secondary indexes
[--gsi=name:hkey{ field_name }[,rkey{ field_name },proj_type{ ALL\|KEYS_ONLY\|INCLUDE }]...] | Allows for the declaration of secondary indexes
[--required=field1...] | A list of attributes that are required for an instance of the model
[--length-validations=field1:MIN-MAX...] | Validations on the length of attributes in a model
[--table-name=name] | Sets the name of the table in DynamoDB, if different than the model name
[--skip-table-config] | Doesn't generate a table config for the model
[--password-digest] | Adds a password field (note that you must have bcrypt has a dependency) that automatically hashes and manages the model password

The included rake task `aws_record:migrate` will run all of the migrations in `app/db/table_config`
| Command Option Names | Purpose |
|----------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------|
| [--skip-namespace], [--no-skip-namespace] | Skip namespace (affects only isolated applications) |
| [--disable-mutation-tracking], [--no-disable-mutation-tracking] | Disables dirty tracking |
| [--timestamps], [--no-timestamps] | Adds created, updated timestamps to the model |
| --table-config=primary:R-W [SecondaryIndex1:R-W]... | Declares the r/w units for the model as well as any secondary indexes |
| [--gsi=name:hkey{ field_name }[,rkey{ field_name },proj_type{ ALL\|KEYS_ONLY\|INCLUDE }]...] | Allows for the declaration of secondary indexes |
| [--required=field1...] | A list of attributes that are required for an instance of the model |
| [--length-validations=field1:MIN-MAX...] | Validations on the length of attributes in a model |
| [--table-name=name] | Sets the name of the table in DynamoDB, if different than the model name |
| [--skip-table-config] | Doesn't generate a table config for the model |
| [--password-digest] | Adds a password field (note that you must have bcrypt has a dependency) that automatically hashes and manages the model password |
2 changes: 0 additions & 2 deletions lib/aws-sdk-rails.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@

require_relative 'action_dispatch/session/dynamo_db_store' if defined?(Aws::SessionStore::DynamoDB::RackMiddleware)

require_relative 'generators/aws_record/base'

module Aws
module Rails
VERSION = File.read(File.expand_path('../VERSION', __dir__)).strip
Expand Down
2 changes: 1 addition & 1 deletion lib/aws/rails/railtie.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ class Railtie < ::Rails::Railtie

rake_tasks do
load 'tasks/dynamo_db/session_store.rake' if defined?(Aws::SessionStore::DynamoDB)
load 'tasks/aws_record/migrate.rake'
load 'tasks/aws_record/migrate.rake' if defined?(Aws::Record)
end
end

Expand Down
Loading