Skip to content

Commit

Permalink
Datacite Action (#833)
Browse files Browse the repository at this point in the history
* scaffold datacite action

* basic deposit

* fix broken deposits

* fix arcadia seed

* added a few tests

* dry

* test tweaks

* format

* lockfile

* fix eslint

* support doi suffix

* fix typeerror

* small fixes and some error messages

* remove log

* suffix test

* publish doi instead drafting

* add lastModifiedBy to updatePub

* fix type error in test

* add datacite related environment variable definitions to terraform

* declare existence of temp datacite secrets

* fix tf issues

* use datacite test api for now

* Update core/actions/datacite/action.ts

Co-authored-by: Thomas F. K. Jorna <[email protected]>

* pr feedback

* fix bad merge

* new values

* fix tests

---------

Co-authored-by: Eric McDaniel <[email protected]>
Co-authored-by: Thomas F. K. Jorna <[email protected]>
  • Loading branch information
3 people authored Dec 19, 2024
1 parent 327147f commit bc6c85d
Show file tree
Hide file tree
Showing 40 changed files with 2,268 additions and 217 deletions.
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/bug-issue.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
name: Bug Report
about: Find something broken or odd? Report it here.
title: ''
title: ""
labels: bug
assignees: ''
assignees: ""
---

<!--
Expand Down
1 change: 1 addition & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
## Issue(s) Resolved

## High-level Explanation of PR

<!-- Using which methods does this PR resolve the issues above? -->

## Test Plan
Expand Down
21 changes: 12 additions & 9 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,21 @@

Docker-compose provides a means to run all the code components as containers. It has these advantages:

- more realistically emulating the setting where this code is run in production.
- less contamination of environment, so spurious failures (or successes) can be avoided.
- easy to boot from nothing without system dependencies except Docker
- more realistically emulating the setting where this code is run in production.
- less contamination of environment, so spurious failures (or successes) can be avoided.
- easy to boot from nothing without system dependencies except Docker

With disadvantages:

- doesn't support hot-reloading
- slightly slower iteration due to `docker build` between runs
- doesn't support hot-reloading
- slightly slower iteration due to `docker build` between runs

With these properties, this is useful for end-to-end tests and locally verifying that the code works when removed from some of the fast-iteration features of `next.js`, such as
JIT compilation. Because a `docker build` is needed to build the containers to run, hot-reloading is not available in this environment; so faster iteration
with `pnpm dev` is recommended until you are ready to run battery of tests or need to verify behavior in an isolated environment.

A slightly modified version of `.env.local` is required, where we remove the DATABASE_URL (this is built out of parts in docker envs):

```
grep -v DATABASE_URL \
<./core/.env.local \
Expand All @@ -27,13 +28,13 @@ grep -v DATABASE_URL \
This cluster will address the local supabase and postgres just like when you are running with `pnpm dev`, so no need to take extra steps for migrations (though the same ones are needed).

To run the full cluster as local docker-compose, first initialize supabase as you would for development; then do:

```
docker compose -f docker-compose.dev.yml up
```

you can now address on `localhost:3000` as before. note that `pnpm dev` uses the same ports and cannot be running at the same time.


## Prettier

At the moment, the repo simply uses prettier before adding any additional complexity with ESLint configs. Just auto-format (either on save, or on commit), and let the .prettierrc hold the small subset of decisions.
Expand All @@ -58,7 +59,6 @@ We currently have a race condition where dev will sometimes fail because we can'

`core` depends on `ui` which depends on `utils`. `utils` often takes longer to build than it does for `ui` to start building, which causes an error to be thrown because `utils` d.ts file has been cleared out during its build and hasn't been replaced yet. This generates an error, but is quick to resolve, so doesn't break actual dev work from beginning. It does make the console output messier though.


## Building and deploying for AWS environments

All change management to Knowledge Futures' production environment is done through github actions.
Expand All @@ -69,6 +69,7 @@ including all details for a container. We don't want to tie code releases to ter
but the service "declaration" relies on this Task Definition to exist.

Therefore based on community patterns we have seen, the flow is roughly this:

1. The infrastructure code in terraform declares a "template" Task Definition.
2. Terraform is told not to change the "service" based on changes to the Task Definition.
3. Any changes to the template will be picked up by the next deploy, which is done outside of Terraform.
Expand Down Expand Up @@ -116,10 +117,11 @@ act \

**AWS CLI access in `act`:**
When you setup `act` locally for the first time, you can choose whether to do a Small, Medium, or Large install.
The Large install is *very large*, but the medium install does not include the AWS CLI.
The Large install is _very large_, but the medium install does not include the AWS CLI.

If you choose to work with the medium install, you will need to customize afterward by editing your `~/.actrc` file
to use an image that includes the AWS CLI, for example your .actrc might look like:

```
-P ubuntu-latest=eveships/act-plusplus:latest
-P ubuntu-22.04=catthehacker/ubuntu:full-22.04
Expand All @@ -140,4 +142,5 @@ AWS_SECRET_ACCESS_KEY...
Images tagged with a SHA alone should be idempotently built, but `-dirty` can be changed/overwritten.

**TODO:**
- [ ] allow deploying without a rebuild, so that a rollback is convenient

- [ ] allow deploying without a rebuild, so that a rollback is convenient
37 changes: 25 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,20 +9,22 @@ As noted in [our recent announcement](https://www.knowledgefutures.org/updates/p
Learn More:
[Documentation](https://help.knowledgefutures.org) | [PubPub Platform](https://knowledgefutures.org/pubpub) | [Knowledge Futures](https://www.knowledgefutures.org/) | [Newsletter](https://pubpub.us5.list-manage.com/subscribe?u=9b9b78707f3dd62d0d47ec03d&id=be26e45660) | [Roadmap](https://github.com/orgs/pubpub/projects/46/views/1)

**PubPub Platform is currently an alpha release, which means the code is subject to frequent, breaking changes. We do not yet recommend running PubPub Platform for production projects.**
**PubPub Platform is currently an alpha release, which means the code is subject to frequent, breaking changes. We do not yet recommend running PubPub Platform for production projects.**

## Community Guidelines and Code of Conduct

Knowledge Futures intends to foster an open and welcoming environment that aligns with our [core values of ACCESS](https://notes.knowledgefutures.org/pub/cqih29xa#our-values-access) (accessibility, collaboration, curiosity, equity, and systemic outlike). As such, we require that all employees and members of our open-source community adhere to our [Code of Conduct](https://github.com/knowledgefutures/general/blob/master/CODE_OF_CONDUCT.md) in all interactions in this and other Knowledge Futures repositories, including issues, discussions, comments, pull requests, commit messages, code, and all other messages.

[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-ff69b4.svg)](https://github.com/knowledgefutures/general/blob/master/CODE_OF_CONDUCT.md)

## Repository Structure

This repo is built as a monorepo that holds first-party components of PubPub. There are four primary sections:

```
root
├── core/
├── infrastructure/
├── infrastructure/
├── integrations/ (deprecated)
├── jobs/
├── packages/
Expand All @@ -38,6 +40,7 @@ root
To avoid inconsistencies and difficult-to-track errors, we specify a particular version of node in `/.nvmrc` (currently `v20.17.0`). We recommend using [nvm](https://github.com/nvm-sh/nvm) to ensure you're using the same version.

## Local Installation

This package runs the version of node specified in `.nvmrc` (currently `v20.17.0`) and uses pnpm for package management. All following commands are run from the root of this package.

To get started, clone the repository and install the version of node specified in `.nvmrc` (we recommend using [nvm](https://github.com/nvm-sh/nvm).
Expand All @@ -56,6 +59,7 @@ pnpm build
Depending on which app or package you are doing work on, you may need to create a .env.local file. See each package's individual README.md file for further details.

## Development

To run all packages in the monorepo workspace, simply run:

```
Expand All @@ -80,47 +84,56 @@ pnpm --filter core migrate-dev
```

## Self-Hosting and Deployment

Guidance for self-hosting and deployment is coming soon. In the meantime, you can read about how we currently deploy the app in [DEVELOPMENT.md](https://github.com/pubpub/platform/blob/main/DEVELOPMENT.md). With small modifications, most development teams can use the included terraform configuration to run the app on commodity cloud infrastructure of their choice.

## Bugs, Feature Requests, Help, and Feedback

We use the [Discussion Forum](https://github.com/pubpub/platform/discussions) for feature requests, ideas, and general feedback, and [GitHub Issues](https://github.com/pubpub/platform/issues/) for day-to-day development. Thus, if you're unsure where to post your feedback, start with a discussion. We can always transfer it to an issue later if needed.

### Bugs

If you have a specific bug to report, feel free to add a [new bug issue](https://github.com/pubpub/platform/issues/new?assignees=&labels=bug&projects=&template=bug-issue.md&title=) to the PubPub Platform Repo. If you submit a bug, we ask that you use the available template and fill it out to the best of your ability, including information about your browser and operating system, detailed, written step-by-step instructions to reproduce the bug and screenshots or a screen recording when relevant. Having all of this information up-front helps us solve any issues faster.

Please search the [issue list](https://github.com/pubpub/platform/issues) first to make sure your bug hasn't already been reported. If it has, add your feedback to the preexisting issue as a comment.

### Feature Requests, Feedback, and Help

If you have a feature request, idea, general feedback, or need help with PubPub, we'd love you to post a discussion on the [Discussion Forum](https://github.com/pubpub/platform/discussions). As with bug reports, make sure to search the forum first to see if the community has already discussed your idea or solved your issue. If we have, feel free to join in on that ongoing discussion. Remember to be polite and courteous. All activity on this repository is governed by the [Knowledge Futures Code of Conduct](https://github.com/knowledgefutures/general/blob/master/CODE_OF_CONDUCT.md).

## Contributing

In the coming weeks, we'll be developing more thorough contribution guides, particularly for contributors interested in:
- Extending PubPub Platform with new Actions and Rules
- Extending the PubPub Platform API
- Contributing to self-hosting scripts and guides on common cloud hosting
- Contributing documentation for developers or users

- Extending PubPub Platform with new Actions and Rules
- Extending the PubPub Platform API
- Contributing to self-hosting scripts and guides on common cloud hosting
- Contributing documentation for developers or users

For now, you can browse the [issue list](https://github.com/pubpub/platform/issues) and comment on any issues you may want
to take on. We'll be in touch shortly
to take on. We'll be in touch shortly

### Pull Requests

Our preferred practice is for contributors to create a branch using the format `initials/descriptive-name` and submit it against main.

Request names should be prefixed with one of the following categories:

- fix: for commits focused on specific bug fixes
- feature: for commits that introduce a new feature
- update: for commits that improve an existing feature
- dev: for commits that focus solely on documentation, refactoring code, or developer experience updates
- fix: for commits focused on specific bug fixes
- feature: for commits that introduce a new feature
- update: for commits that improve an existing feature
- dev: for commits that focus solely on documentation, refactoring code, or developer experience updates

Request descriptions should use to our Pull Request template, including a clear rationale for the PR, listing any issues resolved, and describing the test plan for the request, including both tests you wrote and step-by-step descriptions of any manual QA that may be needed.

Finally, we request that any complex code, new terminology, potentially decisions you made, or any areas you'd like feedback on be commented on inline in GitHub's files changed interface.

## User-Facing Documentation
User-facing documentation is a work in progress, and can be found at https://help.knowledgefutures.org.

User-facing documentation is a work in progress, and can be found at https://help.knowledgefutures.org.

## Supporting Services

Thank you to these groups for providing their tools for free to PubPub's open source mission.

[![Browserstack-logo@2x](https://user-images.githubusercontent.com/1000455/64237395-318a4c80-cef4-11e9-8b78-98ed3ec58ce3.png)](https://www.browserstack.com/)
4 changes: 4 additions & 0 deletions core/.env.development
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,7 @@ HONEYCOMB_API_KEY="xxx"
ARTILLERY_CLOUD_API_KEY="xxx"

KYSELY_DEBUG="true"

DATACITE_REPOSITORY_ID=""
DATACITE_PASSWORD=""
DATACITE_API_URL="https://api.test.datacite.org"
1 change: 1 addition & 0 deletions core/actions/_lib/resolvePubfields.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,5 +58,6 @@ export const resolveWithPubfields = <T extends Record<string, any>>(
return {
...rest,
...pv,
pubFields,
};
};
2 changes: 2 additions & 0 deletions core/actions/api/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import type * as z from "zod";
import type { Event } from "db/public";

import { pubEnteredStage, pubInStageForDuration, pubLeftStage } from "../_lib/rules";
import * as datacite from "../datacite/action";
import * as email from "../email/action";
import * as googleDriveImport from "../googleDriveImport/action";
import * as http from "../http/action";
Expand All @@ -21,6 +22,7 @@ export const actions = {
[http.action.name]: http.action,
[move.action.name]: move.action,
[googleDriveImport.action.name]: googleDriveImport.action,
[datacite.action.name]: datacite.action,
} as const;

export const getActionByName = <N extends keyof typeof actions>(name: N) => {
Expand Down
92 changes: 92 additions & 0 deletions core/actions/datacite/action.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
import * as z from "zod";

import { Action } from "db/public";
import { Globe } from "ui/icon";

import { defineAction } from "../types";

export const action = defineAction({
name: Action.datacite,
config: {
schema: z.object({
doi: z.string().optional(),
doiPrefix: z.string().optional(),
doiSuffix: z.string().optional(),
title: z.string().optional(),
url: z.string(),
publisher: z.string(),
publicationDate: z.date(),
creator: z.string(),
creatorName: z.string(),
}),
fieldConfig: {
doi: {
allowedSchemas: true,
},
doiSuffix: {
allowedSchemas: true,
},
title: {
allowedSchemas: true,
},
url: {
allowedSchemas: true,
},
publisher: {
allowedSchemas: true,
},
publicationDate: {
allowedSchemas: true,
},
creator: {
allowedSchemas: true,
},
creatorName: {
allowedSchemas: true,
},
},
},
params: {
schema: z.object({
doi: z.string().optional(),
doiPrefix: z.string().optional(),
doiSuffix: z.string().optional(),
title: z.string().optional(),
url: z.string(),
publisher: z.string(),
publicationDate: z.date(),
creator: z.string(),
creatorName: z.string(),
}),
fieldConfig: {
doi: {
allowedSchemas: true,
},
doiSuffix: {
allowedSchemas: true,
},
title: {
allowedSchemas: true,
},
url: {
allowedSchemas: true,
},
publisher: {
allowedSchemas: true,
},
publicationDate: {
allowedSchemas: true,
},
creator: {
allowedSchemas: true,
},
creatorName: {
allowedSchemas: true,
},
},
},
description: "Deposit a pub to DataCite",
icon: Globe,
experimental: true,
superAdminOnly: true,
});
Loading

0 comments on commit bc6c85d

Please sign in to comment.