Skip to content

Commit

Permalink
Allow adding STL files to 3D scene (#3242)
Browse files Browse the repository at this point in the history
* allow adding STL files to 3D scene

* add stl import to docs

* update changelog
  • Loading branch information
philippotto authored Sep 21, 2018
1 parent 76565dd commit f72ee2f
Show file tree
Hide file tree
Showing 8 changed files with 314 additions and 10 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ For upgrade instructions, please check the [migration guide](MIGRATIONS.md).
- Added bar-chart visualization to project progress report. [#3224](https://github.com/scalableminds/webknossos/pull/3224)
- Added a button to collapse all comments. [#3215](https://github.com/scalableminds/webknossos/pull/3215)
- The datasets in the dashboard are now sorted according to their user-specific usage. As a result, relevant datasets should appear at the top of the list. [#3206](https://github.com/scalableminds/webknossos/pull/3206)
- 3D Meshes can now be imported into the tracing view by uploading corresponding STL files. [#3242](https://github.com/scalableminds/webknossos/pull/3242)

### Changed

Expand Down
252 changes: 252 additions & 0 deletions app/assets/javascripts/libs/parse_stl_buffer.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,252 @@
/* eslint-disable */

import * as THREE from "three";

/**
* @author aleeper / http://adamleeper.com/
* @author mrdoob / http://mrdoob.com/
* @author gero3 / https://github.com/gero3
* @author Mugen87 / https://github.com/Mugen87
*
* Description: A THREE loader for STL ASCII files, as created by Solidworks and other CAD programs.
*
* Supports both binary and ASCII encoded files, with automatic detection of type.
*
* The loader returns a non-indexed buffer geometry.
*
* Limitations:
* Binary decoding supports "Magics" color format (http://en.wikipedia.org/wiki/STL_(file_format)#Color_in_binary_STL).
* There is perhaps some question as to how valid it is to always assume little-endian-ness.
* ASCII decoding assumes file is UTF-8.
*
* Usage:
* parse(buffer, ( geometry ) => {
* scene.add( new THREE.Mesh( geometry ) );
* });
*
* For binary STLs geometry might contain colors for vertices. To use it:
* // use the same code to load STL as above
* if (geometry.hasColors) {
* material = new THREE.MeshPhongMaterial({ opacity: geometry.alpha, vertexColors: THREE.VertexColors });
* } else { .... }
* var mesh = new THREE.Mesh( geometry, material );
*/

export default function parse(data) {
function isBinary(data) {
var expect, face_size, n_faces, reader;
reader = new DataView(data);
face_size = (32 / 8) * 3 + (32 / 8) * 3 * 3 + 16 / 8;
n_faces = reader.getUint32(80, true);
expect = 80 + 32 / 8 + n_faces * face_size;

if (expect === reader.byteLength) {
return true;
}

// An ASCII STL data must begin with 'solid ' as the first six bytes.
// However, ASCII STLs lacking the SPACE after the 'd' are known to be
// plentiful. So, check the first 5 bytes for 'solid'.

// Several encodings, such as UTF-8, precede the text with up to 5 bytes:
// https://en.wikipedia.org/wiki/Byte_order_mark#Byte_order_marks_by_encoding
// Search for "solid" to start anywhere after those prefixes.

// US-ASCII ordinal values for 's', 'o', 'l', 'i', 'd'

var solid = [115, 111, 108, 105, 100];

for (var off = 0; off < 5; off++) {
// If "solid" text is matched to the current offset, declare it to be an ASCII STL.

if (matchDataViewAt(solid, reader, off)) return false;
}

// Couldn't find "solid" text at the beginning; it is binary STL.

return true;
}

function matchDataViewAt(query, reader, offset) {
// Check if each byte in query matches the corresponding byte from the current offset

for (var i = 0, il = query.length; i < il; i++) {
if (query[i] !== reader.getUint8(offset + i, false)) return false;
}

return true;
}

function parseBinary(data) {
var reader = new DataView(data);
var faces = reader.getUint32(80, true);

var r,
g,
b,
hasColors = false,
colors;
var defaultR, defaultG, defaultB, alpha;

// process STL header
// check for default color in header ("COLOR=rgba" sequence).

for (var index = 0; index < 80 - 10; index++) {
if (
reader.getUint32(index, false) == 0x434f4c4f /*COLO*/ &&
reader.getUint8(index + 4) == 0x52 /*'R'*/ &&
reader.getUint8(index + 5) == 0x3d /*'='*/
) {
hasColors = true;
colors = [];

defaultR = reader.getUint8(index + 6) / 255;
defaultG = reader.getUint8(index + 7) / 255;
defaultB = reader.getUint8(index + 8) / 255;
alpha = reader.getUint8(index + 9) / 255;
}
}

var dataOffset = 84;
var faceLength = 12 * 4 + 2;

var geometry = new THREE.BufferGeometry();

var vertices = [];
var normals = [];

for (var face = 0; face < faces; face++) {
var start = dataOffset + face * faceLength;
var normalX = reader.getFloat32(start, true);
var normalY = reader.getFloat32(start + 4, true);
var normalZ = reader.getFloat32(start + 8, true);

if (hasColors) {
var packedColor = reader.getUint16(start + 48, true);

if ((packedColor & 0x8000) === 0) {
// facet has its own unique color

r = (packedColor & 0x1f) / 31;
g = ((packedColor >> 5) & 0x1f) / 31;
b = ((packedColor >> 10) & 0x1f) / 31;
} else {
r = defaultR;
g = defaultG;
b = defaultB;
}
}

for (var i = 1; i <= 3; i++) {
var vertexstart = start + i * 12;

vertices.push(reader.getFloat32(vertexstart, true));
vertices.push(reader.getFloat32(vertexstart + 4, true));
vertices.push(reader.getFloat32(vertexstart + 8, true));

normals.push(normalX, normalY, normalZ);

if (hasColors) {
colors.push(r, g, b);
}
}
}

geometry.addAttribute("position", new THREE.BufferAttribute(new Float32Array(vertices), 3));
geometry.addAttribute("normal", new THREE.BufferAttribute(new Float32Array(normals), 3));

if (hasColors) {
geometry.addAttribute("color", new THREE.BufferAttribute(new Float32Array(colors), 3));
geometry.hasColors = true;
geometry.alpha = alpha;
}

return geometry;
}

function parseASCII(data) {
var geometry = new THREE.BufferGeometry();
var patternFace = /facet([\s\S]*?)endfacet/g;
var faceCounter = 0;

var patternFloat = /[\s]+([+-]?(?:\d*)(?:\.\d*)?(?:[eE][+-]?\d+)?)/.source;
var patternVertex = new RegExp("vertex" + patternFloat + patternFloat + patternFloat, "g");
var patternNormal = new RegExp("normal" + patternFloat + patternFloat + patternFloat, "g");

var vertices = [];
var normals = [];

var normal = new THREE.Vector3();

var result;

while ((result = patternFace.exec(data)) !== null) {
var vertexCountPerFace = 0;
var normalCountPerFace = 0;

var text = result[0];

while ((result = patternNormal.exec(text)) !== null) {
normal.x = parseFloat(result[1]);
normal.y = parseFloat(result[2]);
normal.z = parseFloat(result[3]);
normalCountPerFace++;
}

while ((result = patternVertex.exec(text)) !== null) {
vertices.push(parseFloat(result[1]), parseFloat(result[2]), parseFloat(result[3]));
normals.push(normal.x, normal.y, normal.z);
vertexCountPerFace++;
}

// every face have to own ONE valid normal

if (normalCountPerFace !== 1) {
console.error(
"THREE.STLLoader: Something isn't right with the normal of face number " + faceCounter,
);
}

// each face have to own THREE valid vertices

if (vertexCountPerFace !== 3) {
console.error(
"THREE.STLLoader: Something isn't right with the vertices of face number " + faceCounter,
);
}

faceCounter++;
}

geometry.addAttribute("position", new THREE.Float32BufferAttribute(vertices, 3));
geometry.addAttribute("normal", new THREE.Float32BufferAttribute(normals, 3));

return geometry;
}

function ensureString(buffer) {
if (typeof buffer !== "string") {
return THREE.LoaderUtils.decodeText(new Uint8Array(buffer));
}

return buffer;
}

function ensureBinary(buffer) {
if (typeof buffer === "string") {
var array_buffer = new Uint8Array(buffer.length);
for (var i = 0; i < buffer.length; i++) {
array_buffer[i] = buffer.charCodeAt(i) & 0xff; // implicitly assumes little-endian
}
return array_buffer.buffer || array_buffer;
} else {
return buffer;
}
}

// start

var binData = ensureBinary(data);

return isBinary(binData) ? parseBinary(binData) : parseASCII(ensureString(data));
}
25 changes: 25 additions & 0 deletions app/assets/javascripts/libs/read_file.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// @flow

export function readFileAsText(file: File): Promise<string> {
return new Promise((resolve, reject) => {
const reader = new FileReader();
reader.onerror = reject;
reader.onload = () => resolve(reader.result.toString());
reader.readAsText(file);
});
}

export function readFileAsArrayBuffer(file: File): Promise<ArrayBuffer> {
return new Promise((resolve, reject) => {
const reader = new FileReader();
reader.onerror = reject;
reader.onload = () => {
if (typeof reader.result === "string") {
// Satisfy flow
throw new Error("Couldn't read buffer");
}
resolve(reader.result);
};
reader.readAsArrayBuffer(file);
});
}
9 changes: 9 additions & 0 deletions app/assets/javascripts/oxalis/controller/scene_controller.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import app from "app";
import * as Utils from "libs/utils";
import BackboneEvents from "backbone-events-standalone";
import * as THREE from "three";
import parseStlBuffer from "libs/parse_stl_buffer";
import { V3 } from "libs/mjs";
import {
getPosition,
Expand Down Expand Up @@ -81,6 +82,14 @@ class SceneController {
this.scene.add(this.rootGroup);
}

addSTL(stlBuffer: ArrayBuffer): void {
const geometry = parseStlBuffer(stlBuffer);
geometry.computeVertexNormals();

const meshMaterial = new THREE.MeshNormalMaterial();
this.scene.add(new THREE.Mesh(geometry, meshMaterial));
}

createMeshes(): void {
this.rootNode = new THREE.Object3D();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import React, { PureComponent } from "react";
import Model from "oxalis/model";
import Store from "oxalis/store";
import { connect } from "react-redux";
import { Button, Dropdown, Menu, Icon, Modal } from "antd";
import { Upload, Button, Dropdown, Menu, Icon, Modal } from "antd";
import Constants from "oxalis/constants";
import MergeModalView from "oxalis/view/action-bar/merge_modal_view";
import ShareModalView from "oxalis/view/action-bar/share_modal_view";
Expand All @@ -20,6 +20,8 @@ import type { OxalisState, RestrictionsAndSettingsType, TaskType } from "oxalis/
import type { APIUserType, APITracingType } from "admin/api_flow_types";
import { layoutEmitter } from "oxalis/view/layouting/layout_persistence";
import { updateUserSettingAction } from "oxalis/model/actions/settings_actions";
import SceneController from "oxalis/controller/scene_controller";
import { readFileAsArrayBuffer } from "libs/read_file";

type StateProps = {
tracingType: APITracingType,
Expand Down Expand Up @@ -265,6 +267,20 @@ class TracingActionsView extends PureComponent<StateProps, State> {

elements.push(resetLayoutItem);

const onStlUpload = async info => {
const buffer = await readFileAsArrayBuffer(info.file);
SceneController.addSTL(buffer);
};

elements.push(
<Menu.Item key="stl-mesh">
<Upload beforeUpload={() => false} onChange={onStlUpload} showUploadList={false}>
<Icon type="upload" />
Import STL Mesh
</Upload>
</Menu.Item>,
);

const menu = <Menu>{elements}</Menu>;

return (
Expand Down
10 changes: 1 addition & 9 deletions app/assets/javascripts/oxalis/view/right-menu/trees_tab_view.js
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import {
setActiveTreeAction,
addTreesAndGroupsAction,
} from "oxalis/model/actions/skeletontracing_actions";
import { readFileAsText } from "libs/read_file";
import Store from "oxalis/store";
import { serializeToNml, getNmlName, parseNml } from "oxalis/model/helpers/nml_helpers";
import * as Utils from "libs/utils";
Expand Down Expand Up @@ -64,15 +65,6 @@ type State = {
isDownloading: boolean,
};

function readFileAsText(file: File): Promise<string> {
return new Promise((resolve, reject) => {
const reader = new FileReader();
reader.onerror = reject;
reader.onload = () => resolve(reader.result.toString());
reader.readAsText(file);
});
}

export async function importNmls(files: Array<File>, createGroupForEachFile: boolean) {
try {
const { successes: importActions, errors } = await Utils.promiseAllWithErrors(
Expand Down
Binary file added docs/images/stl_mesh.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
9 changes: 9 additions & 0 deletions docs/tracing_ui.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ The most common buttons are:
- `Share`: Create a shareable link to your dataset containing the current position, rotation, zoom level etc. Use this to collaboratively work with colleagues. Read more about this feature in the [Sharing guide](./sharing.md).
- `Add Script`: Using the [webKnossos frontend API](https://demo.webknossos.org/assets/docs/frontend-api/index.html) users can interact with webKnossos programmatically. User scripts can be executed from here. Admins can add often used scripts to webKnossos to make them available to all users for easy access.
- `Restore Older Version`: Only available for skeleton tracings. Opens a view that shows all previous versions of a skeleton tracing. From this view, any older version can be selected, previewed, and restored.
- `Import STL Mesh`: 3D Meshes can be imported into the current tracing view by uploading corresponding STL files. Read more information in [Mesh Visualization](#mesh-visualization).

A user can directly jump to positions within their datasets by entering them in the position input field.
The same is true for the rotation in some tracing modes.
Expand Down Expand Up @@ -209,6 +210,14 @@ In the `Segmentation` tab on the right-hand side, you can see the cell IDs which
![Adding labels with the Brush tool](./images/volume_brush.gif)
![Removing labels with the Brush tool](./images/volume_delete.gif)

### Mesh Visualization
With the help of external tools, such as [Amira](https://www.fei.com/software/amira-avizo/), volume data can be converted to 3D meshes. These meshes can be imported into webKnossos to view them alongside the actual data.

To import an STL file, use the `Import STL Mesh` option in the [toolbar](#the-toolbar).

![A 3D Mesh visualized in webKnossos](./images/stl_mesh.png)


## Tracing UI Settings
The settings menu allows users to fine-tune some parameters of webKnossos.
All settings are automatically saved as part of a user's profile.
Expand Down

0 comments on commit f72ee2f

Please sign in to comment.