Skip to content

Commit

Permalink
Merge branch 'main' into aking/2434/new-manual-add-design
Browse files Browse the repository at this point in the history
  • Loading branch information
allisonking committed Feb 24, 2023
2 parents b7709ba + 72b5dfe commit 9c66f38
Show file tree
Hide file tree
Showing 191 changed files with 3,595 additions and 741 deletions.
6 changes: 6 additions & 0 deletions .fides/db_dataset.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1025,6 +1025,12 @@ dataset:
- name: opt_in
data_categories: [system.operations]
data_qualifier: aggregated.anonymized.unlinked_pseudonymized.pseudonymized.identified
- name: has_gpc_flag
data_categories: [ system.operations ]
data_qualifier: aggregated.anonymized.unlinked_pseudonymized.pseudonymized.identified
- name: conflicts_with_gpc
data_categories: [ system.operations ]
data_qualifier: aggregated.anonymized.unlinked_pseudonymized.pseudonymized.identified
- name: provided_identity_id
data_categories: [system.operations]
data_qualifier: aggregated.anonymized.unlinked_pseudonymized.pseudonymized.identified
Expand Down
2 changes: 1 addition & 1 deletion .fides/fides.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ analytics_opt_out = false

[redis]
host = "redis"
password = "testpassword"
password = "redispassword"
port = 6379
charset = "utf8"
default_ttl_seconds = 604800
Expand Down
77 changes: 71 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
<<<<<<< HEAD
=======
# Changelog

>>>>>>> main
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/)
Expand All @@ -16,24 +21,42 @@ The types of changes are:
## [Unreleased](https://github.com/ethyca/fides/compare/2.7.0...main)

### Added

* Add API support for messaging config properties [#2551](https://github.com/ethyca/fides/pull/2551)
* Access and erasure support for Kustomer [#2520](https://github.com/ethyca/fides/pull/2520)
* Added the `erase_after` field on collections to be able to set the order for erasures [#2619](https://github.com/ethyca/fides/pull/2619)

### Changed

- Admin UI
- Add flow for selecting system types when manually creating a system [#2530](https://github.com/ethyca/fides/pull/2530)
- Updated forms for privacy declarations [#2648](https://github.com/ethyca/fides/pull/2648)
- Delete flow for privacy declarations [#2664](https://github.com/ethyca/fides/pull/2664)
* Admin UI
* Add flow for selecting system types when manually creating a system [#2530](https://github.com/ethyca/fides/pull/2530)
* Updated forms for privacy declarations [#2648](https://github.com/ethyca/fides/pull/2648)
* Delete flow for privacy declarations [#2664](https://github.com/ethyca/fides/pull/2664)
* Convert all config values to Pydantic `Field` objects [#2613](https://github.com/ethyca/fides/pull/2613)
* Add warning to 'fides deploy' when installed outside of a virtual environment [#2641](https://github.com/ethyca/fides/pull/2641)
* Change how config creation/import is handled across the application [#2622](https://github.com/ethyca/fides/pull/2622)

- Add warning to 'fides deploy' when installed outside of a virtual environment [#2641](https://github.com/ethyca/fides/pull/2641)
### Developer Experience

* Set the security environment of the fides dev setup to `prod` instead of `dev` [#2588](https://github.com/ethyca/fides/pull/2588)
* Removed unexpected default Redis password [#2666](https://github.com/ethyca/fides/pull/2666)
* Privacy Center
* Typechecking and validation of the `config.json` will be checked for backwards-compatibility. [#2661](https://github.com/ethyca/fides/pull/2661)

### Fixed

* Fix support for "redis.user" setting when authenticating to the Redis cache [#2666](https://github.com/ethyca/fides/pull/2666)
* Fix error with the classify dataset feature flag not writing the dataset to the server [#2675](https://github.com/ethyca/fides/pull/2675)
* Admin UI
* Remove Identifiability (Data Qualifier) from taxonomy editor [2684](https://github.com/ethyca/fides/pull/2684)

## [2.7.0](https://github.com/ethyca/fides/compare/2.6.6...2.7.0)

* Fides API
* Access and erasure support for Braintree [#2223](https://github.com/ethyca/fides/pull/2223)
* Added route to send a test message [#2585](https://github.com/ethyca/fides/pull/2585)
* Add default storage configuration functionality and associated APIs [#2438](https://github.com/ethyca/fides/pull/2438)

* Admin UI
* Custom Metadata [#2536](https://github.com/ethyca/fides/pull/2536)
* Create Custom Lists
Expand All @@ -49,9 +72,15 @@ The types of changes are:

### Added

<<<<<<< HEAD
- Added new Wunderkind Consent Saas Connector [#2600](https://github.com/ethyca/fides/pull/2600)
- Added new Sovrn Email Consent Connector [#2543](https://github.com/ethyca/fides/pull/2543/)
- Log Fides version at startup [#2566](https://github.com/ethyca/fides/pull/2566)
=======
* Added new Wunderkind Consent Saas Connector [#2600](https://github.com/ethyca/fides/pull/2600)
* Added new Sovrn Email Consent Connector [#2543](https://github.com/ethyca/fides/pull/2543/)
* Log Fides version at startup [#2566](https://github.com/ethyca/fides/pull/2566)
>>>>>>> main
### Changed

Expand All @@ -73,11 +102,18 @@ The types of changes are:

### Developer Experience

<<<<<<< HEAD
- Added new Cypress E2E smoke tests [#2241](https://github.com/ethyca/fides/pull/2241)
- Set the security environment of the fides dev setup to `prod` instead of `dev` [#2588](https://github.com/ethyca/fides/pull/2588)
- New command `nox -s e2e_test` which will spin up the test environment and run true E2E Cypress tests against it [#2417](https://github.com/ethyca/fides/pull/2417)
- Cypress E2E tests now run in CI and are reported to Cypress Cloud [#2417](https://github.com/ethyca/fides/pull/2417)
- Change from `randomint` to `uuid` in mongodb tests to reduce flakiness. [#2591](https://github.com/ethyca/fides/pull/2591)
=======
* Added new Cypress E2E smoke tests [#2241](https://github.com/ethyca/fides/pull/2241)
* New command `nox -s e2e_test` which will spin up the test environment and run true E2E Cypress tests against it [#2417](https://github.com/ethyca/fides/pull/2417)
* Cypress E2E tests now run in CI and are reported to Cypress Cloud [#2417](https://github.com/ethyca/fides/pull/2417)
* Change from `randomint` to `uuid` in mongodb tests to reduce flakiness. [#2591](https://github.com/ethyca/fides/pull/2591)
>>>>>>> main
### Removed

Expand Down Expand Up @@ -133,6 +169,7 @@ The types of changes are:

### Added

<<<<<<< HEAD
- Added the `env` option to the `security` configuration options to allow for users to completely secure the API endpoints [#2267](https://github.com/ethyca/fides/pull/2267)
- Unified Fides Resources
- Added a dataset dropdown selector when configuring a connector to link an existing dataset to the connector configuration. [#2162](https://github.com/ethyca/fides/pull/2162)
Expand All @@ -152,6 +189,28 @@ The types of changes are:
- Patch Google Analytics Consent Connector to delete by client_id [#2355](https://github.com/ethyca/fides/pull/2355)
- Add a "skip_param_values option" to optionally skip when we are missing param values in the body [#2384](https://github.com/ethyca/fides/pull/2384)
- Adds a new Universal Analytics Connector that works with the UA Tracking Id
=======
* Added the `env` option to the `security` configuration options to allow for users to completely secure the API endpoints [#2267](https://github.com/ethyca/fides/pull/2267)
* Unified Fides Resources
* Added a dataset dropdown selector when configuring a connector to link an existing dataset to the connector configuration. [#2162](https://github.com/ethyca/fides/pull/2162)
* Added new datasetconfig.ctl_dataset_id field to unify fides dataset resources [#2046](https://github.com/ethyca/fides/pull/2046)
* Add new connection config routes that couple them with systems [#2249](https://github.com/ethyca/fides/pull/2249)
* Add new select/deselect all permissions buttons [#2437](https://github.com/ethyca/fides/pull/2437)
* Endpoints to allow a user with the `user:password-reset` scope to reset users' passwords. In addition, users no longer require a scope to edit their own passwords. [#2373](https://github.com/ethyca/fides/pull/2373)
* New form to reset a user's password without knowing an old password [#2390](https://github.com/ethyca/fides/pull/2390)
* Approve & deny buttons on the "Request details" page. [#2473](https://github.com/ethyca/fides/pull/2473)
* Consent Propagation
* Add the ability to execute Consent Requests via the Privacy Request Execution layer [#2125](https://github.com/ethyca/fides/pull/2125)
* Add a Mailchimp Transactional Consent Connector [#2194](https://github.com/ethyca/fides/pull/2194)
* Allow defining a list of opt-in and/or opt-out requests in consent connectors [#2315](https://github.com/ethyca/fides/pull/2315)
* Add a Google Analytics Consent Connector for GA4 properties [#2302](https://github.com/ethyca/fides/pull/2302)
* Pass the GA Cookie from the Privacy Center [#2337](https://github.com/ethyca/fides/pull/2337)
* Rename "user_id" to more specific "ga_client_id" [#2356](https://github.com/ethyca/fides/pull/2356)
* Patch Google Analytics Consent Connector to delete by client_id [#2355](https://github.com/ethyca/fides/pull/2355)
* Add a "skip_param_values option" to optionally skip when we are missing param values in the body [#2384](https://github.com/ethyca/fides/pull/2384)
* Adds a new Universal Analytics Connector that works with the UA Tracking Id
* Adds intake and storage of Global Privacy Control Signal props for Consent [#2599](https://github.com/ethyca/fides/pull/2599)
>>>>>>> main
### Changed

Expand All @@ -173,9 +232,15 @@ The types of changes are:

### Developer Experience

<<<<<<< HEAD
- `nox -s test_env` has been replaced with `nox -s "fides_env(dev)"`
- New command `nox -s "fides_env(test)"` creates a complete test environment with seed data (similar to `fides_env(dev)`) but with the production fides image so the built UI can be accessed at `localhost:8080` [#2399](https://github.com/ethyca/fides/pull/2399)
- Change from code climate to codecov for coverage reporting [#2402](https://github.com/ethyca/fides/pull/2402)
=======
* `nox -s test_env` has been replaced with `nox -s "fides_env(dev)"`
* New command `nox -s "fides_env(test)"` creates a complete test environment with seed data (similar to `fides_env(dev)`) but with the production fides image so the built UI can be accessed at `localhost:8080` [#2399](https://github.com/ethyca/fides/pull/2399)
* Change from code climate to codecov for coverage reporting [#2402](https://github.com/ethyca/fides/pull/2402)
>>>>>>> main
### Fixed

Expand Down
19 changes: 0 additions & 19 deletions clients/admin-ui/cypress/e2e/taxonomy.cy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ describe("Taxonomy management page", () => {
cy.getByTestId("tab-Data Categories");
cy.getByTestId("tab-Data Uses");
cy.getByTestId("tab-Data Subjects");
cy.getByTestId("tab-Identifiability");
});

describe("Can view data", () => {
Expand All @@ -26,8 +25,6 @@ describe("Taxonomy management page", () => {
cy.wait("@getDataUses");
cy.getByTestId("tab-Data Subjects").click();
cy.wait("@getDataSubjects");
cy.getByTestId("tab-Identifiability").click();
cy.wait("@getDataQualifiers");
cy.getByTestId("tab-Data Categories").click();
cy.wait("@getDataCategories");
});
Expand Down Expand Up @@ -123,16 +120,6 @@ describe("Taxonomy management page", () => {
isParent: false,
request: "@putDataSubject",
},
{
tab: "Identifiability",
name: "Aggregated Data",
key: "aggregated",
description:
"Statistical data that does not contain individually identifying information but includes information about groups of individuals that renders individual identification impossible.",
parentKey: "",
isParent: true,
request: "@putDataQualifier",
},
];
expectedTabValues.forEach((tabValue) => {
cy.getByTestId(`tab-${tabValue.tab}`).click();
Expand Down Expand Up @@ -367,11 +354,6 @@ describe("Taxonomy management page", () => {
name: "Data subject",
request: "@postDataSubject",
},
{
tab: "Identifiability",
name: "Data qualifier",
request: "@postDataQualifier",
},
];
expectedTabValues.forEach((tabValue) => {
cy.getByTestId(`tab-${tabValue.tab}`).click();
Expand Down Expand Up @@ -510,7 +492,6 @@ describe("Taxonomy management page", () => {
const tabValues = [
{ tab: "Data Categories", request: "@deleteDataCategory" },
{ tab: "Data Uses", request: "@deleteDataUse" },
{ tab: "Identifiability", request: "@deleteDataQualifier" },
];
tabValues.forEach((tabValue) => {
cy.getByTestId(`tab-${tabValue.tab}`).click();
Expand Down
2 changes: 1 addition & 1 deletion clients/admin-ui/src/features/dataset/helpers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ export const getUpdatedDatasetFromClassifyDataset = (
draftCollection.name
);

if (classifyCollection?.name !== activeCollection) {
if (activeCollection && classifyCollection?.name !== activeCollection) {
return;
}

Expand Down
11 changes: 1 addition & 10 deletions clients/admin-ui/src/features/taxonomy/TaxonomyTabs.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,7 @@ import { Box, Button } from "@fidesui/react";
import { useAppDispatch } from "~/app/hooks";

import DataTabs, { TabData } from "../common/DataTabs";
import {
useDataCategory,
useDataQualifier,
useDataSubject,
useDataUse,
} from "./hooks";
import { useDataCategory, useDataSubject, useDataUse } from "./hooks";
import { setIsAddFormOpen } from "./taxonomy.slice";
import TaxonomyTabContent from "./TaxonomyTabContent";

Expand All @@ -25,10 +20,6 @@ const TABS: TabData[] = [
label: "Data Subjects",
content: <TaxonomyTabContent useTaxonomy={useDataSubject} />,
},
{
label: "Identifiability",
content: <TaxonomyTabContent useTaxonomy={useDataQualifier} />,
},
];
const TaxonomyTabs = () => {
const dispatch = useAppDispatch();
Expand Down
47 changes: 0 additions & 47 deletions clients/admin-ui/src/features/taxonomy/hooks.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ import { useCustomFields } from "~/features/common/custom-fields/hooks";
import { RTKResult } from "~/features/common/types";
import {
DataCategory,
DataQualifier,
DataSubject,
DataSubjectRightsEnum,
DataUse,
Expand All @@ -23,12 +22,6 @@ import {
CustomTextInput,
} from "../common/form/inputs";
import { enumToOptions } from "../common/helpers";
import {
useCreateDataQualifierMutation,
useDeleteDataQualifierMutation,
useGetAllDataQualifiersQuery,
useUpdateDataQualifierMutation,
} from "../data-qualifier/data-qualifier.slice";
import {
useCreateDataSubjectMutation,
useDeleteDataSubjectMutation,
Expand Down Expand Up @@ -468,43 +461,3 @@ export const useDataSubject = (): TaxonomyHookData<DataSubject> => {
transformEntityToInitialValues,
};
};

export const useDataQualifier = (): TaxonomyHookData<DataQualifier> => {
const { data, isLoading } = useGetAllDataQualifiersQuery();
const [entityToEdit, setEntityToEdit] = useState<DataQualifier | null>(null);

const labels = {
fides_key: "Data qualifier",
name: "Data qualifier name",
description: "Data qualifier description",
parent_key: "Parent data qualifier",
};

const [createDataQualifierMutationTrigger] = useCreateDataQualifierMutation();
const [updateDataQualifierMutationTrigger] = useUpdateDataQualifierMutation();
const [deleteDataQualifierMutationTrigger] = useDeleteDataQualifierMutation();

const handleCreate = (initialValues: FormValues, newValues: FormValues) =>
createDataQualifierMutationTrigger(
transformBaseFormValuesToEntity(initialValues, newValues)
);

const handleEdit = (initialValues: FormValues, newValues: FormValues) =>
updateDataQualifierMutationTrigger(
transformBaseFormValuesToEntity(initialValues, newValues)
);

const handleDelete = deleteDataQualifierMutationTrigger;

return {
data,
isLoading,
labels,
entityToEdit,
setEntityToEdit,
handleCreate,
handleEdit,
handleDelete,
transformEntityToInitialValues: transformTaxonomyBaseToInitialValues,
};
};
27 changes: 27 additions & 0 deletions clients/privacy-center/__tests__/config/examples.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import { Config } from "~/types/config";
import minimalJson from "~/config/examples/minimal.json";
import basicJson from "~/config/examples/basic.json";
import fullJson from "~/config/examples/full.json";

describe("The Config type", () => {
/**
* If this test is failing, it's because the Config type has been updated in a way that is not
* backwards-compatible. To fix this, modify the type so that it is more permissive with what it
* allows, for example by making a property optional. Then you must handle the old/new
* compatibility at runtime with sensible defaults.
*
* DO NOT make this test pass by updating any example JSON files. Those files are real snapshots
* of configurations the PC should support.
*
* Discussion: https://github.com/ethyca/fides/discussions/2392
*/
it("is backwards-compatible with old JSON files", () => {
const minimalTyped: Config = minimalJson;
const basicTyped: Config = basicJson;
const fullTyped: Config = fullJson;

expect(minimalTyped).toBeDefined();
expect(basicTyped).toBeDefined();
expect(fullTyped).toBeDefined();
});
});
41 changes: 41 additions & 0 deletions clients/privacy-center/__tests__/config/validate-config.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
import { produce } from "immer";

import { configIsValid } from "~/scripts/validate-config";
import minimalJson from "~/config/examples/minimal.json";
import fullJson from "~/config/examples/full.json";

describe("configIsValid", () => {
const testCases = [
{
name: "no consent options",
config: minimalJson,
expected: {
isValid: true,
},
},
{
name: "valid consent options",
config: fullJson,
expected: {
isValid: true,
},
},
{
name: "multiple executable consent options",
config: produce(fullJson, (draftConfig) => {
draftConfig.consent.consentOptions[0].executable = true;
draftConfig.consent.consentOptions[1].executable = true;
}),
expected: {
isValid: false,
message: "Cannot have more than one consent option be executable",
},
},
];

testCases.forEach((tc) => {
test(tc.name, () => {
expect(configIsValid(tc.config)).toMatchObject(tc.expected);
});
});
});
Loading

0 comments on commit 9c66f38

Please sign in to comment.