DocAI Expense Parser Demo

Objective

Learn how to use Google Cloud Platform to construct a pipeline to process expenses (ie. receipts). This repo serves as a sample code to build your own demo but is not tested for production.

Visualizing the workflow

GCP Services used in the Demo

• Google Cloud Procurement Document AI
• Google Cloud Storage
• Google Cloud Functions
• BigQuery

Steps to re-create this demo in your own GCP environment

1. Create a Google Cloud Platform Project

2. Enable the Cloud Document AI API, Cloud functions API, and Cloud Build API in the project you created in step #1

3. If you do not have access to the parser, request access via this link. Here is a link to the official Expense Parser documentation.

4. Create a service account that will later be used by Cloud Functions

   1. Navigate to IAM & Admin -> Service Accounts
   2. Click on Create a service account
   3. In the Service account name section, type in process-receipt-example or a name of your choice
   4. Click Create and continue
   5. Grant this service account the following roles:
      • Storage Admin
      • BigQuery Admin
      • Document AI API User
   6. Click Done and you should see this service account in the IAM main page

5. Create your Doc AI processor

   • At this point, you should have your request in Step 3 approved and have access to expense parser
   • Navigate to console -> Document AI -> processors
   • Click Create processor and choose expense parser
   • Name your processor and click Create
   • Take note of your processor's region (eg. us) and processor ID

6. Activate your Command Shell and clone this GitHub Repo in your Command shell using the command:

gh repo clone jiya-zhang/docai-expense-parser-demo
git checkout -b v1api

7. Execute Bash shell scripts in your Cloud Shell terminal to create cloud resources (i.e Google Cloud Storage Buckets, Pub/Sub topics, Cloud Functions, BigQuery dataset and table)

   1. Change directory to the scripts folder

      cd docai-expense-parser-demo
      

   2. Update the following values in .env.local:

      • PROJECT_ID should match your current project's ID
      • BUCKET_LOCATION is where you want the raw receipts to be stored
      • CLOUD_FUNCTION_LOCATION is where your code executes
      • CLOUD_FUNCTION_SERVICE_ACCOUNT should be the same name you created in Step 4

      vim .env.local
      

   3. Make your .sh files executable

      chmod +x set-up-pipeline.sh
      

   4. Change directory to the cloud functions folder

      cd cloud-functions
      

   5. Update the following values in .env.yaml (from your note in Step 5):

      • PARSER_LOCATION
      • PROCESSOR_ID

      vim .env.yaml
      

   6. Go back to the original folder and execute your .sh files to create cloud resources

      cd ..
      ./set-up-pipeline.sh
      

8. Testing/Validating the demo

   1. Upload a sample receipt in the input bucket (<project_id>-input-receipts)
   2. At the end of the processing, you should expect your BigQuery tables to be populated with extracted entities (eg. total_amount, supplier_name, etc.)
   3. With the structured data in BigQuery, we can now design downstream analytical tools to gain actionable insights as well as detect errors/frauds.