[Release] 1.0.0-beta6 (#102)

* mfcc rst and sc help file * mfcc sc examples * mfcc output * mfcc helpfile readability * BufMFCC udpates * update bufonsetslice docs slightly (#87) * fix audio transport segmentation of discussion and desc * fix audio transport segmentation of discussion and desc (#88) * add missing see alsos * Json driver (#35) * Make jinja rendering conditional on driver having a template * minimal driver for raw JSON output * cmake target for raw JSON * fix cmake output directory * add command line option for raw * remove spurious item from raw driver * add back changes that dealt with conditional template rendering * add a github action for creating a basic API on actions Co-authored-by: James Bradbury <[email protected]> * Update json_api.yml * Update json_api.yml * Update json_api.yml * change name of raw output to api.json * Update json_api.yml * remove indentation on api * typo in UMAP * [Docs] OnsetSlice and BufOnsetSlice (#91) * update bufonsetslice text * onsetslice update * ted's changes * copy changes to bufonsetslice also * revamp MelBands and BufMelBands RST, update SC example code * *melbands description * added a hyphen * pitch and buf pitch RST * added link to the learn page * edited normalize description * pitch sc help files * BufFlatten RST and SC examples * buf thresh * sc help file * typo in fluid pitch sc example code * bufthresh rst * typo * james' feedback * add .ds_store to gitignore * pa feedback * pa feedback * pa feedback * PA feedback * [Docs] Update Normalize Docs (#76) * update normalize entry * update description * pa's requests Co-authored-by: James Bradbury <[email protected]> * [Docs] Slicer Feature Objects (#101) * add skeleton of bufnoveltyfeature * caveat novelty feature * add onset feature * remove space * add bufampfeature * realtime amp feature * update output of AmpFeature * clean up the language * caveats for bufonset feature * add onset feature * Add JSON generators for Slicers' feature counterparts * rt noveltyfeature * fix various errors * robots in disguise * change feature -> algorithm * cleanup noveltyfeature * make noveltyfeature more consistent * remove superfluous spaces * fix description * fix reference to buffer version in rt object * add padding * sc example code for slicer-features
flucoma · Mar 30, 2022 · fb28246 · fb28246
1 parent 7dd5ffc
commit fb28246
Show file tree

Hide file tree

Showing 53 changed files with 1,574 additions and 595 deletions.
diff --git a/.github/workflows/json_api.yml b/.github/workflows/json_api.yml
@@ -0,0 +1,48 @@
+name: Dev API
+
+on:
+  workflow_dispatch:
+  push:
+    branches: [ dev, ci/** ]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - uses: flucoma/actions/env@v4
+      - uses: flucoma/actions/docs@v4
+        with:
+          target: 'MAKE_RAW_REF'
+
+      - uses: dev-drprasad/[email protected]
+        with:
+          delete_release: true # default: false
+          tag_name: api # tag name to delete
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Setup SSH
+        uses: shimataro/ssh-key-action@v2
+        with:
+          key: ${{ secrets.SSH }}
+          known_hosts: 'placeholder'
+
+      - name: Adding Known Hosts
+        run: |
+          mkdir -p ~/.ssh
+          ssh-keyscan -H ${{ secrets.DO_IP }} >> ~/.ssh/known_hosts
+
+      - name: package and upload
+        uses: softprops/action-gh-release@v1
+        with:
+          name: FluCoMa Docs API
+          body: "A JSON that can be accessed to derive parameters/args for objects. The build hash is ${{ github.sha }}"
+          files: 'build/flucoma.api.json'
+          prerelease: true
+          tag_name: api
+          draft: false
+
+      - name: Upload To Droplet
+        run: |
+          rsync -avz --delete --no-times --no-o --no-g --no-perms ./build/api.json james@${{ secrets.DO_IP }}:/home/learn/www/
diff --git a/.gitignore b/.gitignore
@@ -10,3 +10,4 @@ schelp
 *.pyc
 interfaces/*
 *.log
+**/.DS_Store
diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -27,6 +27,7 @@ set(MAX_DOC_OUT "${CMAKE_BINARY_DIR}/max_ref" CACHE PATH "")
 set(PD_DOC_OUT  "${CMAKE_BINARY_DIR}/pd_ref" CACHE PATH "")
 set(CLI_DOC_OUT  "${CMAKE_BINARY_DIR}/cli_ref" CACHE PATH "") 
 set(SC_DOC_OUT  "${CMAKE_BINARY_DIR}/sc_ref" CACHE PATH "") 
+set(RAW_DOC_OUT  "${CMAKE_BINARY_DIR}" CACHE PATH "") 
 # option(TESTS,"Run tests?")
 include(FetchContent)
 
@@ -138,6 +139,7 @@ add_ref_target(max "Making Max ref")
 add_ref_target(pd "Making PD ref")
 add_ref_target(cli "Making CLI ref")
 add_ref_target(sc "Making SC ref")
+add_ref_target(raw "Making raw API JSON")
 
 enable_testing()
 

diff --git a/doc/AmpFeature.rst b/doc/AmpFeature.rst
@@ -0,0 +1,41 @@
+:digest: Realtime Amplitude Differential Feature
+:species: transformer
+:sc-categories: Libraries>FluidDecomposition
+:sc-related: Guides/FluidCorpusManipulationToolkit
+:see-also: AmpGate, AmpSlice, OnsetFeature, NoveltyFeature
+:description: Calculate the amplitude differential feature in realtime.
+:discussion: 
+    :fluid-obj:`AmpSlice` uses the differential between a fast and a slow envelope follower to determine changes in amplitude. This object calculates the amplitude differential and copies it to an output buffer.
+
+:process: The audio rate in, control rate out version of the object.
+:output: A KR signal of the feature.
+
+:control in:
+
+    The audio to be processed.
+
+:control fastRampUp:
+
+   The number of samples the relative envelope follower will take to reach the next value when raising. Typically, this will be faster than slowRampUp.
+
+:control fastRampDown:
+
+   The number of samples the relative envelope follower will take to reach the next value when falling. Typically, this will be faster than slowRampDown.
+
+:control slowRampUp:
+
+   The number of samples the absolute envelope follower will take to reach the next value when raising.
+
+:control slowRampDown:
+
+   The number of samples the absolute envelope follower will take to reach the next value when falling.
+
+:control floor:
+
+   The level in dB the slowRamp needs to be above to consider a detected difference valid, allowing to ignore the slices in the noise floor.
+
+:control highPassFreq:
+
+   The frequency of the fourth-order Linkwitz–Riley high-pass filter (https://en.wikipedia.org/wiki/Linkwitz%E2%80%93Riley_filter). This is done first on the signal to minimise low frequency intermodulation with very fast ramp lengths. A frequency of 0 bypasses the filter.
+
+
diff --git a/doc/AudioTransport.rst b/doc/AudioTransport.rst
@@ -2,13 +2,15 @@
 :species: transformer[2]
 :sc-categories: FluidManipulation
 :sc-related: Classes/FluidBufAudioTransport
-:see-also: 
+:see-also: NMFMorph, BufNMFCross
 :description: 
-   Interpolates between the spectra of two sounds using the Optimal Transport algorithm
+   Interpolates between the spectra of two sounds using optimal transport.
 
+:discussion:
+   Interpolates between the spectra of two sounds using the optimal transport algorithm. This enables morphing and hybridisation of the perceptual qualities of each source linearly.
    See Henderson and Solomonm (2019) AUDIO TRANSPORT: A GENERALIZED PORTAMENTO VIA OPTIMAL TRANSPORT, DaFx
 
-
+   https://arxiv.org/abs/1906.06763
 
 :control in2:
 

diff --git a/doc/BufAmpFeature.rst b/doc/BufAmpFeature.rst
@@ -0,0 +1,67 @@
+:digest: Buffer-Based Amplitude Differential Feature
+:species: buffer-proc
+:sc-categories: Libraries>FluidDecomposition
+:sc-related: Guides/FluidCorpusManipulationToolkit
+:see-also: BufAmpSlice, BufNoveltyFeature, BufAmpFeature, BufOnsetFeature
+:description: Calculate the amplitude differential feature used by :fluid-obj:`BufAmpSlice`.
+:discussion: 
+    :fluid-obj:`BufAmpSlice` uses the differential between a fast and a slow envelope follower to determine changes in amplitude. This object calculates the amplitude differential and copies it to an output buffer.
+
+:output: Nothing, as the destination buffer is declared in the function call.
+
+
+:control source:
+
+   The index of the buffer to use as the source material to be sliced through novelty identification. The different channels of multichannel buffers will be summed.
+
+:control startFrame:
+
+   Where in the srcBuf should the slicing process start, in sample.
+
+:control numFrames:
+
+   How many frames should be processed.
+
+:control startChan:
+
+   For multichannel sources, which channel should be processed.
+
+:control numChans:
+
+   For multichannel sources, how many channel should be summed.
+
+:control features:
+
+   The index of the buffer where the amplitude differential feature will be copied to.
+
+:control fastRampUp:
+
+   The number of samples the relative envelope follower will take to reach the next value when raising. Typically, this will be faster than slowRampUp.
+
+:control fastRampDown:
+
+   The number of samples the relative envelope follower will take to reach the next value when falling. Typically, this will be faster than slowRampDown.
+
+:control slowRampUp:
+
+   The number of samples the absolute envelope follower will take to reach the next value when raising.
+
+:control slowRampDown:
+
+   The number of samples the absolute envelope follower will take to reach the next value when falling.
+
+:control floor:
+
+   The level in dB the slowRamp needs to be above to consider a detected difference valid, allowing to ignore the slices in the noise floor.
+
+:control highPassFreq:
+
+   The frequency of the fourth-order Linkwitz–Riley high-pass filter (https://en.wikipedia.org/wiki/Linkwitz%E2%80%93Riley_filter). This is done first on the signal to minimise low frequency intermodulation with very fast ramp lengths. A frequency of 0 bypasses the filter.
+
+:control padding:
+
+   Controls the zero-padding added to either end of the source buffer or segment. Possible values are 0 (no padding), 1 (default, half the window size), or 2 (window size - hop size). Padding ensures that all input samples are completely analysed: with no padding, the first analysis window starts at time 0, and the samples at either end will be tapered by the STFT windowing function. Mode 1 has the effect of centering the first sample in the analysis window and ensuring that the very start and end of the segment are accounted for in the analysis. Mode 2 can be useful when the overlap factor (window size / hop size) is greater than 2, to ensure that the input samples at either end of the segment are covered by the same number of analysis frames as the rest of the analysed material.
+
+:control action:
+
+   A Function to be evaluated once the offline process has finished and indices instance variables have been updated on the client side. The function will be passed indices as an argument.
diff --git a/doc/BufAudioTransport.rst b/doc/BufAudioTransport.rst
@@ -2,13 +2,16 @@
 :species: buffer-proc
 :sc-categories: FluidManipulation
 :sc-related: Classes/FluidAudioTransport
-:see-also: 
+:see-also: NMFMorph, BufNMFCross
 :description: 
-   Interpolates between the spectra of two sounds using the Optimal Transport algorithm
+   Interpolates between the spectra of two sounds using optimal transport
 
-   See Henderson and Solomonm (2019) AUDIO TRANSPORT: A GENERALIZED PORTAMENTO VIA OPTIMAL TRANSPORT, DaFx
+:discussion:
+   Interpolates between the spectra of two sounds using the optimal transport algorithm. This enables morphing and hybridisation of the perceptual qualities of each source linearly.
 
+   See Henderson and Solomonm (2019) AUDIO TRANSPORT: A GENERALIZED PORTAMENTO VIA OPTIMAL TRANSPORT, DaFx
 
+   https://arxiv.org/abs/1906.06763
 
 :control sourceA:
 

diff --git a/doc/BufFlatten.rst b/doc/BufFlatten.rst
@@ -2,59 +2,56 @@
 :species: buffer-proc
 :sc-categories: FluidCorpusManipulation
 :sc-related: Classes/Buffer
-:see-also: 
+:see-also: BufCompose, BufStats
 :description: 
-   Flatten a multichannel |buffer| to a single channel. This can be useful for constructing n-dimensional data points for use with :fluid-obj:`DataSet`
+   Flatten a multichannel |buffer| to a single channel. This can be useful to structure a buffer such that it can be added to a :fluid-obj:`DataSet`
 
-   The ``axis`` determines how the flattening is arranged. The default value, 1, flattens channel-wise, such that (if we imagine channels are rows, time positions are columns):
+   The ``axis`` argument determines how the flattening is arranged. The default value, 1, flattens channel-wise (similar to how audio files are stored), such that (if we imagine channels are rows, time positions are columns):
 
     ===  ===  ===
     a 1  a 2  a 3
     b 1  b 2  b 3
     c 1  c 2  c 3
     ===  ===  ===
 
-
    becomes
 
     ===  ===  ===  ===  ===  ===  ===  ===  ===
     a 1  b 1  c 1  a 2  b 2  c 2  a 3  b 3  c 3
     ===  ===  ===  ===  ===  ===  ===  ===  ===
 
-
-   whereas with ``axis = 0`` we get
+   whereas with ``axis`` = 0, the buffer is flattened frame-wise, resulting in:
 
     ===  ===  ===  ===  ===  ===  ===  ===  ===
     a 1  a 2  a 3  b 1  b 2  b 3  c 1  c 2  c 3
     ===  ===  ===  ===  ===  ===  ===  ===  ===
-
-
+    
+    To learn more visit https://learn.flucoma.org/reference/bufflatten/.
 
 :control source:
 
-   The |buffer| to flatten
+   The |buffer| to flatten.
 
 :control startFrame:
 
-   Where in the source should the flattening process start, in samples.
+   Where in ``source`` should the flattening process start, in samples. The default is 0.
 
 :control numFrames:
 
-   How many frames should be processed.
+   How many frames should be processed. The default of -1 indicates to process through the end of the buffer.
 
 :control startChan:
 
-   For multichannel source buffers, which channel to start processing at.
+   For multichannel ``source`` buffers, which which channel to begin the processing. The default is 0.
 
 :control numChans:
 
-   For multichannel source buffers, how many channels should be processed.
+   For multichannel ``source`` buffers, how many channels should be processed. The default of -1 indicates to process up through the last channel in  ``source``.
 
 :control destination:
 
-   The |buffer| to write the flattened data to
+   The |buffer| to write the flattened data to.
 
 :control axis:
 
-   Whether to group points channel-wise or frame-wise
-
+   Whether to flatten the buffer channel-wise (1) or frame-wise (0). The default is 1 (channel-wise). 
diff --git a/doc/BufMFCC.rst b/doc/BufMFCC.rst
@@ -4,59 +4,64 @@
 :sc-related: Guides/FluidCorpusManipulationToolkit, Guides/FluidBufMultiThreading, Classes/FluidBufMelBands
 :see-also: 
 :description: 
-   This class implements a classic spectral descriptor, the Mel-Frequency Cepstral Coefficients (https://en.wikipedia.org/wiki/Mel-frequency_cepstrum). The input is first filtered in to **numBands** perceptually-spaced bands, as in Classes/FluidMelBands. It is then analysed into **numCoeffs** number of cepstral coefficients. It has the advantage of being amplitude invariant, except for the first coefficient. It is part of the Guides/FluidCorpusManipulationToolkit of Guides/FluidCorpusManipulationToolkit. For more explanations, learning material, and discussions on its musicianly uses, visit http://www.flucoma.org/
-
-   The process will return a single multichannel buffer of **numCoeffs** per input channel. Each frame represents a value, which is every hopSize.
+
+  MFCC stands for Mel-Frequency Cepstral Coefficients ("cepstral" is pronounced like "kepstral"). This analysis is often used for timbral description and timbral comparison. It compresses the overall spectrum into a smaller number of coefficients that, when taken together, describe the general contour the the spectrum.
 
+  The MFCC values are derived by first computing a mel-frequency spectrum, just as in :fluid-obj:`MelBands`. ``numCoeffs`` coefficients are then calculated by using that mel-frequency spectrum as input to the discrete cosine transform. This means that the shape of the mel-frequency spectrum is compared to a number of cosine wave shapes (different cosines shapes created from different different frequencies). Each MFCC value (i.e., "coefficient") represents how similar the mel-frequency spectrum is to one of these cosine shapes. 
 
+  Other that the 0th coefficient, MFCCs are unchanged by differences in the overall energy of the spectrum (which relates to how we perceive loudness). This means that timbres with similar spectral contours, but different volumes, will still have similar MFCC values, other than MFCC 0. To remove any indication of loudness but keep the information about timbre, we can ignore MFCC 0 by setting the parameter ``startCoeff`` to 1.
 
+   For more information visit https://learn.flucoma.org/reference/mfcc/.
+
+   For an interactive explanation of this relationship, visit https://learn.flucoma.org/reference/mfcc/explain.
+
 :control source:
 
-   The index of the buffer to use as the source material to be described through the various descriptors. The different channels of multichannel buffers will be processing sequentially.
+   The index of the buffer to use as the source material to be analysed. The different channels of multichannel buffers will be processing sequentially.
 
 :control startFrame:
 
-   Where in the srcBuf should the process start, in sample.
+   Where in the ``srcBuf`` the analysis should start, in samples. The default is 0.
 
 :control numFrames:
 
-   How many frames should be processed.
+   How many frames should be analysed. The default of -1 indicates to analyse to the end of the buffer.
 
 :control startChan:
 
-   For multichannel srcBuf, which channel should be processed first.
+   For a multichannel ``srcBuf``, which channel should be processed first. The default is 0.
 
 :control numChans:
 
-   For multichannel srcBuf, how many channel should be processed.
+   For a multichannel ``srcBuf``, how many channels should be processed. The default of -1 indicates to analyse through the last channel.
 
 :control features:
 
-   The destination buffer for the numCoeffs coefficients describing the spectral shape.
+   The destination buffer to write the MFCC analysis into.
 
 :control numCoeffs:
 
-   The number of cepstral coefficients to be outputed. It will decide how many channels are produce per channel of the source.
+   The number of cepstral coefficients to return. The default is 13.
 
 :control numBands:
 
-   The number of bands that will be perceptually equally distributed between **minFreq** and **maxFreq**.
+   The number of bands that will be perceptually equally distributed between ``minFreq`` and ``maxFreq``. The default is 40.
 
 :control startCoeff:
 
-   The lowest index of the output cepstral coefficient, zero-counting.
+   The lowest index of the output cepstral coefficients to return, zero-counting. This can be useful to skip over the 0th coefficient (by indicating ``startCoeff`` = 1), because the 0th coefficient is representative of the overall energy in spectrum, while the rest of the coefficients are not affected by overall energy, only the mel-frequency spectral contour. The default is 0.
 
 :control minFreq:
 
-   The lower boundary of the lowest band of the model, in Hz.
+   The lower bound of the frequency band to use in analysis, in Hz. The default is 20.
 
 :control maxFreq:
 
-   The highest boundary of the highest band of the model, in Hz.
+   The upper bound of the frequency band to use in analysis, in Hz. The default is 20000.
 
 :control windowSize:
 
-   The window size. As MFCC computation relies on spectral frames, we need to decide what precision we give it spectrally and temporally, in line with Gabor Uncertainty principles. http://www.subsurfwiki.org/wiki/Gabor_uncertainty
+   The window size. As MFCC computation relies on spectral frames, we need to decide what precision we give it spectrally and temporally, in line with Gabor Uncertainty principles. http://www.subsurfwiki.org/wiki/Gabor_uncertainty. The default is 1024.
 
 :control hopSize:
 
@@ -68,13 +73,4 @@
 
 :control padding:
 
-   Controls the zero-padding added to either end of the source buffer or segment. Possible values are 0 (no padding), 1 (default, half the window size), or 2 (window size - hop size). Padding ensures that all input samples are completely analysed: with no padding, the first analysis window starts at time 0, and the samples at either end will be tapered by the STFT windowing function. Mode 1 has the effect of centering the first sample in the analysis window and ensuring that the very start and end of the segment are accounted for in the analysis. Mode 2 can be useful when the overlap factor (window size / hop size) is greater than 2, to ensure that the input samples at either end of the segment are covered by the same number of analysis frames as the rest of the analysed material.
-
-:control maxNumCoeffs:
-
-   The maximum number of cepstral coefficients that can be computed. This sets the number of channels of the output, and therefore cannot be modulated.
-
-:control maxFFTSize:
-
-   How large can the FFT be, by allocating memory at instantiation time. This cannot be modulated.
-
+   Controls the zero-padding added to either end of the source buffer or segment. Possible values are 0 (no padding), 1 (default, half the window size), or 2 (window size - hop size). Padding ensures that all input samples are completely analysed: with no padding, the first analysis window starts at time 0, and the samples at either end will be tapered by the STFT windowing function. Mode 1 has the effect of centring the first sample in the analysis window and ensuring that the very start and end of the segment are accounted for in the analysis. Mode 2 can be useful when the overlap factor (window size / hop size) is greater than 2, to ensure that the input samples at either end of the segment are covered by the same number of analysis frames as the rest of the analysed material.
-Original file line number
+Diff line change
@@ Expand Up / @@ -10,3 +10,4 @@ schelp @@
     *.pyc
     interfaces/*
     *.log
+    **/.DS_Store