Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support loading custom metadata blocks #85

Closed
poikilotherm opened this issue Sep 5, 2019 · 4 comments · Fixed by #134
Closed

Support loading custom metadata blocks #85

poikilotherm opened this issue Sep 5, 2019 · 4 comments · Fixed by #134
Labels
enhancement New feature or request
Milestone
v4.17

Comments

@poikilotherm
Copy link
Member

We need to support people having their own custom metadata blocks. See upstream docs on this.

  1. We need to provide a maintenance job to load TSV based metadata blocks
  2. We need a job exporting the (so called) "dynamic) schema, adding it to the Solr instance and reload. (see also upstream docs)
This was referenced Sep 5, 2019
Closed
Open
@poikilotherm poikilotherm added the enhancement New feature or request label Sep 19, 2019
@poikilotherm poikilotherm added this to the v4.17 milestone Sep 19, 2019
@pdurbin
Copy link
Member

pdurbin commented Oct 18, 2019

I would love to use dataverse-kubernetes to spin up Dataverse with the custom metadata block included in IQSS/dataverse#6289 by @doigl

@pdurbin pdurbin mentioned this issue Nov 15, 2019
Closed
@pdurbin
Copy link
Member

pdurbin commented Nov 18, 2019

This was implemented today in IQSS/dataverse-ansible#135 for dataverse-ansible and I put some screenshots from my testing in a comment at IQSS/dataverse-ansible#133 (comment)

@poikilotherm thanks for updateSchemaMDB.sh!

@4tikhonov
Copy link
Collaborator

Great! We're going to test it as well!

@pdurbin
Copy link
Member

pdurbin commented Nov 19, 2019

@4tikhonov perfect! Thanks!

@donsizemore I just gave you and @poikilotherm a shout out in this thread called "custom metadata blocks now easier to spin up and evaluate" - https://groups.google.com/d/msg/dataverse-community/uKretKox_io/4FyPVAMYBgAJ

Thanks again! 🎉 🎉 🎉

@poikilotherm poikilotherm mentioned this issue Jan 9, 2020
Closed
3 tasks
poikilotherm added a commit that referenced this issue Jan 13, 2020
@poikilotherm
Add webhooks for schema sidecar. Relates to #85
d75b96a 
The schema files need to be downloaded from Dataverse and loaded into Solr.
On the other hand, Solr needs to start without those, so a fallback is necessary.

* We modify the schema.xml to include loaded files, but fallback to upstream default.
* We add tini to have an init system for the sidecar
* We add webhook from https://github.com/adnanh/webhook to have a framework to create our webhook
* We add the webhook "update-schema" and its configuration
* We add a startup script with some nice logic around the webhook application
poikilotherm added a commit that referenced this issue Jan 13, 2020
@poikilotherm
Update Solr Kubernetes objects with new sidecar options. See #85.
388651f
@poikilotherm poikilotherm mentioned this issue Jan 13, 2020
Merged
poikilotherm added a commit that referenced this issue Jan 15, 2020
@poikilotherm
Update solr-k8s image with init container script. Add forked update s…
2c90360 
…cript. Related to #85 and #135
poikilotherm added a commit that referenced this issue Jan 15, 2020
@poikilotherm
Add documentation about the Solr schema updating via init and sidecar…
b378162 
…. See #85.
poikilotherm added a commit that referenced this issue Jan 15, 2020
@poikilotherm
Add example to docs how to get custom metadata inside the job. See #85.
8bc091c
T-Haeussermann pushed a commit to HyperSpec-FDM/dataverse-kubernetes that referenced this issue Nov 16, 2023
@poikilotherm @T-Haeussermann
Update solr-k8s image with init container script. Add forked update s…
b57c70e 
…cript. Related to gdcc#85 and gdcc#135
T-Haeussermann pushed a commit to HyperSpec-FDM/dataverse-kubernetes that referenced this issue Nov 16, 2023
@poikilotherm @T-Haeussermann
Add documentation about the Solr schema updating via init and sidecar…
6dbc991 
…. See gdcc#85.
T-Haeussermann pushed a commit to HyperSpec-FDM/dataverse-kubernetes that referenced this issue Nov 16, 2023
@poikilotherm @T-Haeussermann
Add example to docs how to get custom metadata inside the job. See gd…
9e78138 
…cc#85.
T-Haeussermann pushed a commit to HyperSpec-FDM/dataverse-kubernetes that referenced this issue Apr 25, 2024
@poikilotherm @T-Haeussermann
Update solr-k8s image with init container script. Add forked update s…
8b112b7 
…cript. Related to gdcc#85 and gdcc#135
T-Haeussermann pushed a commit to HyperSpec-FDM/dataverse-kubernetes that referenced this issue Apr 25, 2024
@poikilotherm @T-Haeussermann
Add documentation about the Solr schema updating via init and sidecar…
12c03ba 
…. See gdcc#85.
T-Haeussermann pushed a commit to HyperSpec-FDM/dataverse-kubernetes that referenced this issue Apr 25, 2024
@poikilotherm @T-Haeussermann
Add example to docs how to get custom metadata inside the job. See gd…
74af823 
…cc#85.
T-Haeussermann pushed a commit to HyperSpec-FDM/dataverse-kubernetes that referenced this issue Apr 25, 2024
@poikilotherm @T-Haeussermann
Update solr-k8s image with init container script. Add forked update s…
01ff7d3 
…cript. Related to gdcc#85 and gdcc#135
T-Haeussermann pushed a commit to HyperSpec-FDM/dataverse-kubernetes that referenced this issue Apr 25, 2024
@poikilotherm @T-Haeussermann
Add documentation about the Solr schema updating via init and sidecar…
d89068a 
…. See gdcc#85.
T-Haeussermann pushed a commit to HyperSpec-FDM/dataverse-kubernetes that referenced this issue Apr 25, 2024
@poikilotherm @T-Haeussermann
Add example to docs how to get custom metadata inside the job. See gd…
b74ce52 
…cc#85.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
v4.17
Development

Successfully merging a pull request may close this issue.

WIP: 85 metadata blocks gdcc/dataverse-kubernetes
3 participants
@pdurbin @4tikhonov @poikilotherm