Fix #3729, #3730, #3777, #91, #20, part of #3625: Introduce support f…

…or localizing lesson content & multi-lingual answer submission (#3796)

* Add support for AABs, build flavors, and proguard.

There are a lot of details to cover here--see the PR for the complete
context.

* Lint & codeowner fixes.

* Fix failures.

- Add missing codeowner
- Add support for configuring base branch reference
- Update CI for dev/alpha AAB builds to use 'develop' since there's no
  origin configured by default in the workflows

* Different attempt to fix bad develop reference in CI.

* Initial commit.

This is needed to open a PR on GitHub. This commit is being made so that
the PR can start off in a broken Actions state.

This also initially disables most non-Bazel workflows to make workflow
iteration faster and less impacting on other team members.

* Introduce infrastructure for batching.

This introduces a new mechanism for passing lists of tests to sharded
test targets in CI, and hooks it up. No actual sharding is occurring
yet. This led to some simplifications in the CI workflow since the
script can be more dynamic in computing the full list of targets (which
also works around a previous bug with instrumentation tests being run).
Java proto lite also needed to be upgraded for the scripts to be able to
use it.

More testing/documentation needed as this functionality continues to
expand.

* Add bucketing strategy.

This simply partitions bucketed groups of targets into chunks of 10 for
each run. Only 3 buckets are currently retained to test sharding in CI
before introducing full support.

* Fix caching & stabilize builds.

Fixes some caching bucket and output bugs. Also, introduces while loop &
keep_going to introduce resilience against app test build failures (or
just test failures in general).

* Increase sharding & add randomization.

Also, enable other workflows.

Note that CI shouldn't fully pass yet since some documentation and
testing needs to be added yet, but this is meant to be a more realistic
test of the CI environment before the PR is finished.

* Improving partitionin & readability.

Adds a human-readable prefix to make the shards look a bit nicer.

Also, adds more fine-tuned partitioning to bucket & reduce shard counts
to improve overall timing. Will need to be tested in CI.

* Add new tests & fix static analysis errors.

* Fix script.

A newly computed variable wasn't updated to be used in an earlier
change.

* Fix broken tests & test configuration.

Add docstrings for proto.

* Fix mistake from earlier commit.

* Try 10 max parallel actions instead.

See
#3757 (comment)
for context.

* Fix another error from an earlier commit.

* Localisation updates from https://translatewiki.net.

* Fix mv command so it works on Linux & OSX.

Neither 'mv -t' nor piping to mv work on OSX so we needed to find an
alternative (in this case just trying to move everything). This actually
works a bit better since it's doing a per-file move rather than
accommodating for files that shouldn't be moved (which isn't an issue
since the destination directory is different than the one containing the
AAB file).

* Introduce initial domain layer for translations.

Documentation, thorough tests, and detailed description of these changes
are still needed.

* Initial app layer implementation for translations.

This demonstrates working string selection for system-based and
overwritten app languages, including necessary activity recreation &
layout direction overwriting.

This also includes a bunch of Dagger infra refactoring so that some app
layer packages can now be modularized (including the new packages).

* Domain changes needed per downstream UI changes.

* Add patterns & fixes.

This involves MANY broad changes to ensure consistent string retrieval
(for arrays and plurals), formatting, and string transformations
throughout the codebase. Some extra patterns to added to fix things that
were needed, and a few issues were fixed along the way.

* Add needed domain changes for downstream branch.

Also includes fixing circular dependency issue by splitting out some of
the locale components to be part of utility rather than domain (so that
utiltiy and other packages can depend on MachineLocale).

* Introduce support for content localization.

This includes a bunch of stuff that'll be described in more detail in
the PR description, but it essentially:
- Adds support for displaying content in explorations, questions,
  concept cards, and revision cards in a non-English language
- Adds support for submitting non-English answers
- Updates test structures to validate everything exception questions is
  working for localization

* Fix structures to work with parsing assumptions.

* Fix regex checks for translated strings.

Also, performance improvements for the regex check.

* Lint-ish fix.

* Fix failing regex checks.

* Add check for nested res subdirectories.

* Clean up locale infra.

Add some other needed functionality.

* Attempt to delete strings to force history.

* Make AAB builds/runs manual-only targets.

* Fix broken tests.

* Fix lint issues & add KDocs.

Also, abstract ContentLocale for consistency & to disallow direct
construction.

* Add 6/11 test suites (& placeholders for other 4).

Silence one file missing a test suite (since it doesn't need one).

Also, some tweaks to the language support definitions.

* Add more test suites for domain layers.

Included introducing a new general purpose utility for testing data
providers + its own test suite.

* Introduce wrapper & fake for bidi wrapping.

Also, add test version of AssetRepository.

Add new placeholder tests & update all tests project-wide to make sure
that they build.

* Add remaining tests.

Included some shadow refactoring, and introducing new test-only
resources.

* Fix Gradle builds.

* Lint fixes.

* Resolve remaining incomplete TODOs.

* Add new codeowners.

* Post-merge fixes.

Make all non-app layer targets build (haven't run tests yet).

Audited existing bidi wrapping cases & converted strings over to being
%s-only.

* Fix most test targets (builds).

All non-app tests confirmed as passing.

* Fix all remaining test builds.

Introduce new TestActivity for scaffolding all non-activity tests.

* Fix all app layer tests.

Add fixes for question player & old answer displaying.

Add fix for guaranteed crash on startup after some changes between now &
the first build of MR3 (dueu to extra updates in
SplashActivityPresenter).

* Fix questions & profile issues.

* Type specifier pattern & fixes.

Address temporary TODO by removing kdoc.

* Add missing KDocs.

* Boilerplate & TODOs for needed tests.

* Add new needed test dep.

Required an update to truth proto lite import (due to an incompatible
update in the common Truth dep).

* Add needed testing coverage.

Other miscellaneous fixes needed to support new tests.

* Two fixes.

1. Introduce proper API compatibility for LocaleController
2. Ensure TranslationController is scoped (breaks test in downstream PR)

* Fix Gradle builds on branch.

* Resolve nearly all pending TODOs.

Only remainder is a test suite whose tests need to be migrated.

* Lint fixes.

* Re-add method removed from merge.

* Lint fixes.

This also fixes broken extra/unused imports from the merge.

Verified that the dev build works as of this commit. Haven't verified
anything else.

* Fix compute affected tests script.

Adds support for very large PR changesets.

* Fix failures found on CI.

* Fix remaining Gradle failures found in CI.

* Fix existing domain + app layer tests.

Some reworking was needed in QuestionAssessmentProgressControllerTest.

* Post-merge fix.

* Gradle Espresso test fix.

* Add missing KDocs, remove extra file, and other cleanups.

* Lint fixes.

* Deflake DataProviderTestMonitorTest.

* Address reviewer comments.

* Lint fixes.

* Fix affected tests from earlier changes.

These failures were found from CI test workflows.

* Fix remaining Gradle failures.

This introduces a proper fallback mechanism for content strings that
allows Gradle builds & tests to work properly, and adds more robustness
in case misconfigurations actually happen.

* Add placeholders for new needed tests.

* Fix broken tests.

This came from the earlier commit's fix--the suite hadn't been updated.

* Add needed tests for new behaviors.

* Fix Gradle build & mechanism change failures.

* Lint fixes.

* Undo inadvertent change to Gradle jvmargs.

* Disable most tests on Espresso.

* Test fixes + make monitor Espresso-compatible.

* Lint fixes.

Co-authored-by: translatewiki.net <[email protected]>

.gitignore

@@ -30,3 +30,4 @@ config/oppia-dev-workflow-remote-cache-credentials.json
 bazel-*
 .bazelproject
 .aswb
+*.pb

app/BUILD.bazel

@@ -17,7 +17,10 @@ load("@tools_android//tools/googleservices:defs.bzl", "google_services_xml")
 load("//app:app_test.bzl", "app_test")
 load("//app:test_with_resources.bzl", "test_with_resources")
 
-package(default_visibility = ["//utility:__subpackages__"])
+package(default_visibility = [
+    "//domain:__subpackages__",
+    "//utility:__subpackages__",
+])
 
 exports_files(["src/main/AndroidManifest.xml"])
 
@@ -792,6 +795,7 @@ TEST_DEPS = [
     "//app/src/main/java/org/oppia/android/app/translation/testing:test_module",
     "//domain",
     "//testing",
+    "//testing/src/main/java/org/oppia/android/testing/data:data_provider_test_monitor",
     "//testing/src/main/java/org/oppia/android/testing/espresso:edit_text_input_action",
     "//testing/src/main/java/org/oppia/android/testing/espresso:generic_view_matchers",
     "//testing/src/main/java/org/oppia/android/testing/espresso:image_view_matcher",

app/src/main/java/org/oppia/android/app/hintsandsolution/HintsAndSolutionDialogFragment.kt

@@ -10,6 +10,7 @@ import org.oppia.android.app.fragment.FragmentComponentImpl
 import org.oppia.android.app.fragment.InjectableDialogFragment
 import org.oppia.android.app.model.HelpIndex
 import org.oppia.android.app.model.State
+import org.oppia.android.app.model.WrittenTranslationContext
 import org.oppia.android.util.extensions.getProto
 import org.oppia.android.util.extensions.getStringFromBundle
 import org.oppia.android.util.extensions.putProto
@@ -44,25 +45,32 @@ class HintsAndSolutionDialogFragment :
     internal const val ID_ARGUMENT_KEY = "HintsAndSolutionDialogFragment.id"
     internal const val STATE_KEY = "HintsAndSolutionDialogFragment.state"
     internal const val HELP_INDEX_KEY = "HintsAndSolutionDialogFragment.help_index"
+    internal const val WRITTEN_TRANSLATION_CONTEXT_KEY =
+      "HintsAndSolutionDialogFragment.written_translation_context"
 
     /**
      * Creates a new instance of a DialogFragment to display hints and solution
      *
-     * @param id Used in ExplorationController/QuestionAssessmentProgressController to get current state data.
+     * @param id Used in ExplorationController/QuestionAssessmentProgressController to get current
+     *     state data.
      * @param state the [State] being viewed by the learner
      * @param helpIndex the [HelpIndex] corresponding to the current hints/solution configuration
+     * @param writtenTranslationContext the [WrittenTranslationContext] needed to translate the
+     *     hints/solution
      * @return [HintsAndSolutionDialogFragment]: DialogFragment
      */
     fun newInstance(
       id: String,
       state: State,
-      helpIndex: HelpIndex
+      helpIndex: HelpIndex,
+      writtenTranslationContext: WrittenTranslationContext
     ): HintsAndSolutionDialogFragment {
       return HintsAndSolutionDialogFragment().apply {
         arguments = Bundle().apply {
           putString(ID_ARGUMENT_KEY, id)
           putProto(STATE_KEY, state)
           putProto(HELP_INDEX_KEY, helpIndex)
+          putProto(WRITTEN_TRANSLATION_CONTEXT_KEY, writtenTranslationContext)
         }
       }
     }
@@ -107,12 +115,15 @@ class HintsAndSolutionDialogFragment :
 
     val state = args.getProto(STATE_KEY, State.getDefaultInstance())
     val helpIndex = args.getProto(HELP_INDEX_KEY, HelpIndex.getDefaultInstance())
+    val writtenTranslationContext =
+      args.getProto(WRITTEN_TRANSLATION_CONTEXT_KEY, WrittenTranslationContext.getDefaultInstance())
 
     return hintsAndSolutionDialogFragmentPresenter.handleCreateView(
       inflater,
       container,
       state,
       helpIndex,
+      writtenTranslationContext,
       id,
       currentExpandedHintListIndex,
       this as ExpandedHintListIndexListener,

app/src/main/java/org/oppia/android/app/hintsandsolution/HintsAndSolutionDialogFragmentPresenter.kt

@@ -12,6 +12,7 @@ import org.oppia.android.app.model.HelpIndex.IndexTypeCase.LATEST_REVEALED_HINT_
 import org.oppia.android.app.model.HelpIndex.IndexTypeCase.NEXT_AVAILABLE_HINT_INDEX
 import org.oppia.android.app.model.HelpIndex.IndexTypeCase.SHOW_SOLUTION
 import org.oppia.android.app.model.State
+import org.oppia.android.app.model.WrittenTranslationContext
 import org.oppia.android.app.recyclerview.BindableAdapter
 import org.oppia.android.app.translation.AppLanguageResourceHandler
 import org.oppia.android.app.viewmodel.ViewModelProvider
@@ -47,6 +48,7 @@ class HintsAndSolutionDialogFragmentPresenter @Inject constructor(
   private lateinit var binding: HintsAndSolutionFragmentBinding
   private lateinit var state: State
   private lateinit var helpIndex: HelpIndex
+  private lateinit var writtenTranslationContext: WrittenTranslationContext
   private lateinit var itemList: List<HintsAndSolutionItemViewModel>
   private lateinit var bindingAdapter: BindableAdapter<HintsAndSolutionItemViewModel>
 
@@ -63,6 +65,7 @@ class HintsAndSolutionDialogFragmentPresenter @Inject constructor(
     container: ViewGroup?,
     state: State,
     helpIndex: HelpIndex,
+    writtenTranslationContext: WrittenTranslationContext,
     id: String?,
     currentExpandedHintListIndex: Int?,
     expandedHintListIndexListener: ExpandedHintListIndexListener,
@@ -93,6 +96,7 @@ class HintsAndSolutionDialogFragmentPresenter @Inject constructor(
 
     this.state = state
     this.helpIndex = helpIndex
+    this.writtenTranslationContext = writtenTranslationContext
     // The newAvailableHintIndex received here is coming from state player but in this
     // implementation hints/solutions are shown on every even index and on every odd index we show a
     // divider. The relative index therefore needs to be doubled to account for the divider.
@@ -137,7 +141,9 @@ class HintsAndSolutionDialogFragmentPresenter @Inject constructor(
   private fun loadHintsAndSolution(state: State) {
     // Check if hints are available for this state.
     if (state.interaction.hintList.isNotEmpty()) {
-      viewModel.initialize(helpIndex, state.interaction.hintList, state.interaction.solution)
+      viewModel.initialize(
+        helpIndex, state.interaction.hintList, state.interaction.solution, writtenTranslationContext
+      )
 
       itemList = viewModel.processHintList()
 

app/src/main/java/org/oppia/android/app/hintsandsolution/HintsViewModel.kt

@@ -7,9 +7,11 @@ import org.oppia.android.app.fragment.FragmentScope
 import org.oppia.android.app.model.HelpIndex
 import org.oppia.android.app.model.Hint
 import org.oppia.android.app.model.Solution
+import org.oppia.android.app.model.WrittenTranslationContext
 import org.oppia.android.app.translation.AppLanguageResourceHandler
 import org.oppia.android.domain.hintsandsolution.isHintRevealed
 import org.oppia.android.domain.hintsandsolution.isSolutionRevealed
+import org.oppia.android.domain.translation.TranslationController
 import javax.inject.Inject
 
 /**
@@ -24,7 +26,8 @@ private const val DEFAULT_HINT_AND_SOLUTION_SUMMARY = ""
 /** [ViewModel] for Hints in [HintsAndSolutionDialogFragment]. */
 @FragmentScope
 class HintsViewModel @Inject constructor(
-  private val resourceHandler: AppLanguageResourceHandler
+  private val resourceHandler: AppLanguageResourceHandler,
+  private val translationController: TranslationController
 ) : HintsAndSolutionItemViewModel() {
 
   val newAvailableHintIndex = ObservableField<Int>(-1)
@@ -39,13 +42,20 @@ class HintsViewModel @Inject constructor(
   private lateinit var hintList: List<Hint>
   private lateinit var solution: Solution
   private lateinit var helpIndex: HelpIndex
+  private lateinit var writtenTranslationContext: WrittenTranslationContext
   val itemList: MutableList<HintsAndSolutionItemViewModel> = ArrayList()
 
   /** Initializes the view model to display hints and a solution. */
-  fun initialize(helpIndex: HelpIndex, hintList: List<Hint>, solution: Solution) {
+  fun initialize(
+    helpIndex: HelpIndex,
+    hintList: List<Hint>,
+    solution: Solution,
+    writtenTranslationContext: WrittenTranslationContext
+  ) {
     this.helpIndex = helpIndex
     this.hintList = hintList
     this.solution = solution
+    this.writtenTranslationContext = writtenTranslationContext
   }
 
   fun processHintList(): List<HintsAndSolutionItemViewModel> {
@@ -93,9 +103,11 @@ class HintsViewModel @Inject constructor(
   }
 
   private fun addHintToList(hintIndex: Int, hint: Hint) {
-    val hintsViewModel = HintsViewModel(resourceHandler)
+    val hintsViewModel = HintsViewModel(resourceHandler, translationController)
     hintsViewModel.title.set(hint.hintContent.contentId)
-    hintsViewModel.hintsAndSolutionSummary.set(hint.hintContent.html)
+    val hintContentHtml =
+      translationController.extractString(hint.hintContent, writtenTranslationContext)
+    hintsViewModel.hintsAndSolutionSummary.set(hintContentHtml)
     hintsViewModel.isHintRevealed.set(helpIndex.isHintRevealed(hintIndex, hintList))
     itemList.add(hintsViewModel)
     addDividerItem()
@@ -109,7 +121,9 @@ class HintsViewModel @Inject constructor(
     solutionViewModel.denominator.set(solution.correctAnswer.denominator)
     solutionViewModel.wholeNumber.set(solution.correctAnswer.wholeNumber)
     solutionViewModel.isNegative.set(solution.correctAnswer.isNegative)
-    solutionViewModel.solutionSummary.set(solution.explanation.html)
+    val explanationHtml =
+      translationController.extractString(solution.explanation, writtenTranslationContext)
+    solutionViewModel.solutionSummary.set(explanationHtml)
     solutionViewModel.isSolutionRevealed.set(helpIndex.isSolutionRevealed())
     itemList.add(solutionViewModel)
     addDividerItem()

app/src/main/java/org/oppia/android/app/home/recentlyplayed/RecentlyPlayedFragmentPresenter.kt

@@ -241,7 +241,9 @@ class RecentlyPlayedFragmentPresenter @Inject constructor(
     if (promotedStory.chapterPlayState == ChapterPlayState.IN_PROGRESS_SAVED) {
       val explorationCheckpointLiveData =
         explorationCheckpointController.retrieveExplorationCheckpoint(
-          ProfileId.getDefaultInstance(),
+          ProfileId.newBuilder().apply {
+            internalId = internalProfileId
+          }.build(),
           promotedStory.explorationId
         ).toLiveData()
 

app/src/main/java/org/oppia/android/app/player/exploration/ExplorationActivity.kt

@@ -15,6 +15,7 @@ import org.oppia.android.app.hintsandsolution.RevealSolutionInterface
 import org.oppia.android.app.model.HelpIndex
 import org.oppia.android.app.model.ReadingTextSize
 import org.oppia.android.app.model.State
+import org.oppia.android.app.model.WrittenTranslationContext
 import org.oppia.android.app.player.audio.AudioButtonListener
 import org.oppia.android.app.player.state.listener.RouteToHintsAndSolutionListener
 import org.oppia.android.app.player.state.listener.StateKeyboardButtonListener
@@ -45,6 +46,7 @@ class ExplorationActivity :
   private lateinit var storyId: String
   private lateinit var explorationId: String
   private lateinit var state: State
+  private lateinit var writtenTranslationContext: WrittenTranslationContext
   private var backflowScreen: Int? = null
   private var isCheckpointingEnabled: Boolean = false
 
@@ -165,7 +167,8 @@ class ExplorationActivity :
       val hintsAndSolutionDialogFragment = HintsAndSolutionDialogFragment.newInstance(
         explorationId,
         state,
-        helpIndex
+        helpIndex,
+        writtenTranslationContext
       )
       hintsAndSolutionDialogFragment.showNow(supportFragmentManager, TAG_HINTS_AND_SOLUTION_DIALOG)
     }
@@ -179,8 +182,12 @@ class ExplorationActivity :
     explorationActivityPresenter.loadExplorationFragment(readingTextSize)
   }
 
-  override fun onExplorationStateLoaded(state: State) {
+  override fun onExplorationStateLoaded(
+    state: State,
+    writtenTranslationContext: WrittenTranslationContext
+  ) {
     this.state = state
+    this.writtenTranslationContext = writtenTranslationContext
   }
 
   override fun dismissConceptCard() = explorationActivityPresenter.dismissConceptCard()

app/src/main/java/org/oppia/android/app/player/exploration/HintsAndSolutionExplorationManagerFragmentPresenter.kt

@@ -57,7 +57,7 @@ class HintsAndSolutionExplorationManagerFragmentPresenter @Inject constructor(
     // Check if hints are available for this state.
     if (ephemeralState.state.interaction.hintList.size != 0) {
       (activity as HintsAndSolutionExplorationManagerListener).onExplorationStateLoaded(
-        ephemeralState.state
+        ephemeralState.state, ephemeralState.writtenTranslationContext
       )
     }
   }

app/src/main/java/org/oppia/android/app/player/exploration/HintsAndSolutionExplorationManagerListener.kt

@@ -1,8 +1,9 @@
 package org.oppia.android.app.player.exploration
 
 import org.oppia.android.app.model.State
+import org.oppia.android.app.model.WrittenTranslationContext
 
 /** Listener for fetching current exploration state data. */
 interface HintsAndSolutionExplorationManagerListener {
-  fun onExplorationStateLoaded(state: State)
+  fun onExplorationStateLoaded(state: State, writtenTranslationContext: WrittenTranslationContext)
 }

app/src/main/java/org/oppia/android/app/player/state/SelectionInteractionView.kt

@@ -7,6 +7,7 @@ import androidx.databinding.BindingAdapter
 import androidx.fragment.app.Fragment
 import androidx.fragment.app.FragmentManager
 import androidx.recyclerview.widget.RecyclerView
+import org.oppia.android.app.model.WrittenTranslationContext
 import org.oppia.android.app.player.state.itemviewmodel.SelectionInteractionContentViewModel
 import org.oppia.android.app.player.state.itemviewmodel.SelectionItemInputType
 import org.oppia.android.app.recyclerview.BindableAdapter
@@ -45,6 +46,7 @@ class SelectionInteractionView @JvmOverloads constructor(
   lateinit var bindingInterface: ViewBindingShim
 
   private lateinit var entityId: String
+  private lateinit var writtenTranslationContext: WrittenTranslationContext
 
   override fun onAttachedToWindow() {
     super.onAttachedToWindow()
@@ -69,6 +71,15 @@ class SelectionInteractionView @JvmOverloads constructor(
     this.entityId = entityId
   }
 
+  /**
+   * Sets the [WrittenTranslationContext] used to translate strings in this view.
+   *
+   * This must be called during view initialization.
+   */
+  fun setWrittenTranslationContext(writtenTranslationContext: WrittenTranslationContext) {
+    this.writtenTranslationContext = writtenTranslationContext
+  }
+
   private fun createAdapter(): BindableAdapter<SelectionInteractionContentViewModel> {
     return when (selectionItemInputType) {
       SelectionItemInputType.CHECKBOXES ->
@@ -89,7 +100,8 @@ class SelectionInteractionView @JvmOverloads constructor(
                 htmlParserFactory,
                 resourceBucketName,
                 entityType,
-                entityId
+                entityId,
+                writtenTranslationContext
               )
             }
           )
@@ -112,7 +124,8 @@ class SelectionInteractionView @JvmOverloads constructor(
                 htmlParserFactory,
                 resourceBucketName,
                 entityType,
-                entityId
+                entityId,
+                writtenTranslationContext
               )
             }
           )
@@ -127,3 +140,10 @@ fun setEntityId(
   selectionInteractionView: SelectionInteractionView,
   entityId: String
 ) = selectionInteractionView.setEntityId(entityId)
+
+/** Sets the translation context for a specific [SelectionInteractionView] via data-binding. */
+@BindingAdapter("writtenTranslationContext")
+fun setWrittenTranslationContext(
+  selectionInteractionView: SelectionInteractionView,
+  writtenTranslationContext: WrittenTranslationContext
+) = selectionInteractionView.setWrittenTranslationContext(writtenTranslationContext)

app/src/main/java/org/oppia/android/app/player/state/StateFragmentPresenter.kt

@@ -112,7 +112,7 @@ class StateFragmentPresenter @Inject constructor(
       /* attachToRoot= */ false
     )
     recyclerViewAssembler = createRecyclerViewAssembler(
-      assemblerBuilderFactory.create(resourceBucketName, entityType),
+      assemblerBuilderFactory.create(resourceBucketName, entityType, profileId),
       binding.congratulationsTextView,
       binding.congratulationsTextConfettiView,
       binding.fullScreenConfettiView
@@ -275,7 +275,7 @@ class StateFragmentPresenter @Inject constructor(
   private fun subscribeToCurrentState() {
     ephemeralStateLiveData.observe(
       fragment,
-      Observer { result ->
+      { result ->
         processEphemeralStateResult(result)
       }
     )