[CIVIC-5993] Better support for Issued and Updated dataset properties from harvested sources #1802

rhabbachi · 2017-03-15T18:28:36Z

Issue: CIVIC-5993

Description

Currently we are using the $node->created and $node->modified node properties to store the modified and issued. The node properties being subject of change in various stages of the harvest process or other un-related activities this data is being consistently lost.

Acceptance Criteria

Add new fields to the dataset node to store this data ({{field_harvest_source_issued}}, {{field_harvest_source_modified}}, the fields are of type date.
Those fields should not be available on the node edit page.
Those fields should not show up on the node view page. Instead, the value from those fields should supplement the {{Release Date}} and the {{Modified Date}} fields when available.
During the harvest a POD source. Those fields should be filled using the {{modified}} for the {{field_harvest_source_modified}} and {{issued}} for the {{field_harvest_source_issued}} field.
Drop the {{field_modified_source_date}} field as its purpose is being supplemented by the {{field_modified_source_date}} field.
Update the {{open_data_schema_map_dkan}} endpoints to leverage the new date tracking fields from the harvest.

Test Updates

Update the behat tests if needed. Update the phpunit tests to account for this change.

Documentation Updates

Check the documentation and updated when needed.

DKAN Starter Updates

Update the features_master module list to include the date_views module.

Merge process

~~- [ ] Note if there is a special sequence for merging (such as PRs on other repos that need to be merged first) or other abnormalities to look out for.~~

Reminders

There is test for the issue.
CHANGELOG updated.
Coding standards checked.

…aset

…s in dataset view page

Also update the harvest migrate support to accomodate this change.

…aset

+ Use the more generic Date type for the harevst date fields.

+ Fix field name typo.

+ Replace with newly added harvest date tracking fields when appropriate.

…om the harvest fields

dafeder · 2017-03-23T15:31:28Z

modules/dkan/dkan_dataset/dkan_dataset.install

@@ -120,3 +120,14 @@ function dkan_dataset_update_7003() {
  $vocabulary = taxonomy_vocabulary_machine_name_load('odfe_theme');
  taxonomy_vocabulary_delete($vocabulary->vid);
 }
+


Let's move this to dkan.install - we're trying not to do update hooks in the modules anymore

… from harvested sources (#1802) * Add harvest creatde and modified dates to dataset * Support harvest created and modified in endpoints * Use dedicated fields to store created and modified dates * Add tests for harvest date tracking fields in the dataset * Fix source modified field name * Update behat tests to include checking for harvest source dates fields in dataset view page * Fix drush harvest command missing argument warning * Convert the created and modified harvest fields to Date type Also update the harvest migrate support to accomodate this change. * Update harvest dates field names to match Ac + Use the more generic Date type for the harevst date fields. * Update harvest views to use the new date trakcing fields * Update the alter hook to target new harvest date tracking fields * Update unit tests to match the new fields names * Use 'Y-m-d' format for harvest dataset fields * Update test to expect a timestamp instead of date + Fix field name typo. * Fix harvest dataset modified field name typo * Drop field_modified_source_date from code + Replace with newly added harvest date tracking fields when appropriate. * Add hook_update to drop field_modified_source_date from the DB * Override the release and modified in dataset view page with values from the harvest fields * Update the tests to use the new harvest modified field * Update the tests to use the new harvest date fields * Update the api endpoint harvest field names * Update default content file with correct harvest date fields values * Update the exported endpoint to use appropriate time format * Update 1.13.3.rst * Move update_hook to dkan.install * Add CHANGELOG * Update 1.13.3.rst

Riadh Habbachi added 4 commits March 15, 2017 18:42

Add harvest creatde and modified dates to dataset

7d6d5e7

Support harvest created and modified in endpoints

e3a082c

Use dedicated fields to store created and modified dates

087dafd

Add tests for harvest date tracking fields in the dataset

3f4cee4

rhabbachi added the don't merge label Mar 15, 2017

Riadh Habbachi added 2 commits March 16, 2017 07:56

Fix source modified field name

00e07bd

Merge remote-tracking branch 'origin/7.x-1.x' into harvest-fields-dat…

4c00ebb

…aset

rhabbachi removed the don't merge label Mar 16, 2017

Update behat tests to include checking for harvest source dates field…

7ceecc0

…s in dataset view page

rhabbachi force-pushed the harvest-fields-dataset branch from 40e9332 to 7ceecc0 Compare March 16, 2017 09:10

Riadh Habbachi and others added 19 commits March 16, 2017 10:32

Fix drush harvest command missing argument warning

6a36105

Convert the created and modified harvest fields to Date type

3d101a7

Also update the harvest migrate support to accomodate this change.

Merge remote-tracking branch 'origin/7.x-1.x' into harvest-fields-dat…

9bbef67

…aset

Update harvest dates field names to match Ac

e18c190

+ Use the more generic Date type for the harevst date fields.

Update harvest views to use the new date trakcing fields

e1c0550

Update the alter hook to target new harvest date tracking fields

4a9289b

Update unit tests to match the new fields names

47c6077

Use 'Y-m-d' format for harvest dataset fields

731d55c

Update test to expect a timestamp instead of date

953ea78

+ Fix field name typo.

Fix harvest dataset modified field name typo

847b2cf

Drop field_modified_source_date from code

6482821

+ Replace with newly added harvest date tracking fields when appropriate.

Add hook_update to drop field_modified_source_date from the DB

8c18240

Override the release and modified in dataset view page with values fr…

dae3d64

…om the harvest fields

Update the tests to use the new harvest modified field

9375349

Update the tests to use the new harvest date fields

175b360

Update the api endpoint harvest field names

3dfc1e0

Update default content file with correct harvest date fields values

8d2fcb7

Update the exported endpoint to use appropriate time format

dda47db

Update 1.13.3.rst

d4c578e

dafeder requested changes Mar 23, 2017

View reviewed changes

dafeder and others added 4 commits March 23, 2017 12:12

Merge branch '7.x-1.x' into harvest-fields-dataset

ba672e3

Move update_hook to dkan.install

7b4a6cd

Add CHANGELOG

492067d

Update 1.13.3.rst

6357406

dafeder approved these changes Mar 27, 2017

View reviewed changes

Merge branch '7.x-1.x' into harvest-fields-dataset

3245cbf

dafeder merged commit 9752902 into 7.x-1.x Mar 28, 2017

dafeder deleted the harvest-fields-dataset branch March 28, 2017 18:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CIVIC-5993] Better support for Issued and Updated dataset properties from harvested sources #1802

[CIVIC-5993] Better support for Issued and Updated dataset properties from harvested sources #1802

rhabbachi commented Mar 15, 2017 •

edited

Loading

dafeder Mar 23, 2017

[CIVIC-5993] Better support for Issued and Updated dataset properties from harvested sources #1802

[CIVIC-5993] Better support for Issued and Updated dataset properties from harvested sources #1802

Conversation

rhabbachi commented Mar 15, 2017 • edited Loading

Description

Acceptance Criteria

Test Updates

Documentation Updates

DKAN Starter Updates

Merge process

Reminders

dafeder Mar 23, 2017

Choose a reason for hiding this comment

rhabbachi commented Mar 15, 2017 •

edited

Loading