Skip to content

Commit

Permalink
0.3.3 Release
Browse files Browse the repository at this point in the history
0.3.3 Release
  • Loading branch information
lander86 authored May 20, 2022
2 parents 973fff7 + efab05b commit ae1b1ab
Show file tree
Hide file tree
Showing 98 changed files with 2,138 additions and 465 deletions.
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
**0.3.3**
1. Mainchain block deserialization fix: CompactSize usage issue.
2. Bootstrapping tool improvement: scgenesisinfo data parsing.
3. Added logic for checking storages consistency at node startup, and trying to recover the situation for instance if a crash happened during update procedure.
4. CertificateSubmitter on active sync improvement in `getMessageToSign` method.
5. Added HTTP API for stopping the SC node and a hook for calling custom application stop procedure.

**0.3.2**
1. CertificateSubmitter and CertificateSignaturesManager actors restart strategy and failures processing improvement.

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ While we keep monitoring the memory footprint of the proofs generation process,
- After the installation, just run `export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1` before starting the sidechain node, or run the sidechain node adding `LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1` at the beginning of the java command line as follows:

```
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1 java -cp ./target/sidechains-sdk-simpleapp-0.3.2.jar:./target/lib/* com.horizen.examples.SimpleApp <path_to_config_file>
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1 java -cp ./target/sidechains-sdk-simpleapp-0.3.3.jar:./target/lib/* com.horizen.examples.SimpleApp <path_to_config_file>
```
- In the folder `ci` you will find the script `run_sc.sh` to automatically check and use jemalloc library while starting the sidechain node.

Expand Down
2 changes: 1 addition & 1 deletion ci/run_sc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

set -eo pipefail

SIMPLE_APP_VERSION="${SIMPLE_APP_VERSION:-0.3.2}"
SIMPLE_APP_VERSION="${SIMPLE_APP_VERSION:-0.3.3}"

if [ -d "$1" ] && [ -f "$2" ]; then
path_to_jemalloc="$(ldconfig -p | grep "$(arch)" | grep 'libjemalloc\.so\.1$' | tr -d ' ' | cut -d '>' -f 2)"
Expand Down
6 changes: 3 additions & 3 deletions examples/simpleapp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,12 @@ Otherwise, to run SimpleApp outside the IDE:
* (Windows)
```
cd Sidechains-SDK\examples\simpleapp
java -cp ./target/sidechains-sdk-simpleapp-0.3.2.jar;./target/lib/* com.horizen.examples.SimpleApp <path_to_config_file>
java -cp ./target/sidechains-sdk-simpleapp-0.3.3.jar;./target/lib/* com.horizen.examples.SimpleApp <path_to_config_file>
```
* (Linux)
```
cd ./Sidechains-SDK/examples/simpleapp
java -cp ./target/sidechains-sdk-simpleapp-0.3.2.jar:./target/lib/* com.horizen.examples.SimpleApp <path_to_config_file>
java -cp ./target/sidechains-sdk-simpleapp-0.3.3.jar:./target/lib/* com.horizen.examples.SimpleApp <path_to_config_file>
```
On some Linux OSs during backward transfers certificates proofs generation a extremely big RAM consumption may happen, that will lead to the process force killing by the OS.
Expand All @@ -36,7 +36,7 @@ While we keep monitoring the memory footprint of the proofs generation process,
- After the installation, just run `export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1` before starting the sidechain node, or run the sidechain node adding `LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1` at the beginning of the java command line as follows:
```
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1 java -cp ./target/sidechains-sdk-simpleapp-0.3.2.jar:./target/lib/* com.horizen.examples.SimpleApp <path_to_config_file>
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1 java -cp ./target/sidechains-sdk-simpleapp-0.3.3.jar:./target/lib/* com.horizen.examples.SimpleApp <path_to_config_file>
```
- In the folder `ci` you will find the script `run_sc.sh` to automatically check and use jemalloc library while starting the sidechain node.
Expand Down
8 changes: 4 additions & 4 deletions examples/simpleapp/mc_sc_workflow_example.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Build SDK components by using command (in the root of the SDK folder):

Run Bootstrapping tool using command:

`java -jar tools/sctool/target/sidechains-sdk-scbootstrappingtools-0.3.2.jar`
`java -jar tools/sctool/target/sidechains-sdk-scbootstrappingtools-0.3.3.jar`

All other commands are performed as commands for Bootstrapping tool in next format: `"command name" "parameters for command in JSON format"`.
For any help you could use command `help`, for exit just print `exit`
Expand Down Expand Up @@ -397,15 +397,15 @@ Run SimpleApp with the `my_settings.conf`:

* For Windows:
```
java -cp ./examples/simpleapp/target/sidechains-sdk-simpleapp-0.3.2.jar;./examples/simpleapp/target/lib/* com.horizen.examples.SimpleApp ./examples/my_settings.conf
java -cp ./examples/simpleapp/target/sidechains-sdk-simpleapp-0.3.3.jar;./examples/simpleapp/target/lib/* com.horizen.examples.SimpleApp ./examples/my_settings.conf
```
* For Linux (Glibc):
```
java -cp ./examples/simpleapp/target/sidechains-sdk-simpleapp-0.3.2.jar:./examples/simpleapp/target/lib/* com.horizen.examples.SimpleApp ./examples/my_settings.conf
java -cp ./examples/simpleapp/target/sidechains-sdk-simpleapp-0.3.3.jar:./examples/simpleapp/target/lib/* com.horizen.examples.SimpleApp ./examples/my_settings.conf
```
* For Linux (Jemalloc):
```
LD_PRELOAD=<path to jemalloc library>/libjemalloc.so.1 java -cp ./examples/simpleapp/target/sidechains-sdk-simpleapp-0.3.2.jar:./examples/simpleapp/target/lib/* com.horizen.examples.SimpleApp ./examples/my_settings.conf
LD_PRELOAD=<path to jemalloc library>/libjemalloc.so.1 java -cp ./examples/simpleapp/target/sidechains-sdk-simpleapp-0.3.3.jar:./examples/simpleapp/target/lib/* com.horizen.examples.SimpleApp ./examples/my_settings.conf
```
- In the folder `ci` you will find the script `run_sc.sh` to automatically check and use jemalloc library while starting the sidechain node.

Expand Down
4 changes: 2 additions & 2 deletions examples/simpleapp/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>io.horizen</groupId>
<artifactId>sidechains-sdk-simpleapp</artifactId>
<version>0.3.2</version>
<version>0.3.3</version>
<inceptionYear>2018</inceptionYear>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
Expand All @@ -16,7 +16,7 @@
<dependency>
<groupId>io.horizen</groupId>
<artifactId>sidechains-sdk</artifactId>
<version>0.3.2</version>
<version>0.3.3</version>
</dependency>

<dependency>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,35 @@
import com.horizen.proposition.Proposition;
import com.horizen.state.ApplicationState;
import com.horizen.state.SidechainStateReader;
import com.horizen.storage.leveldb.VersionedLevelDbStorageAdapter;
import com.horizen.transaction.BoxTransaction;
import com.horizen.utils.ByteArrayWrapper;
import com.horizen.utils.BytesUtils;
import com.horizen.utils.Pair;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import scala.util.Success;
import scala.util.Try;

import java.util.List;
import java.io.File;
import java.util.*;

public class DefaultApplicationState implements ApplicationState {

// two distinct storages are used in order to test a version misalignment during startup and the recover logic
private VersionedLevelDbStorageAdapter appStorage1;
private VersionedLevelDbStorageAdapter appStorage2;

public static byte[] NULL_VERSION = new byte[32];
public static byte[] DUMMY_KEY = BytesUtils.fromHexString("abcd");

public DefaultApplicationState(File storage1, File storage2) {
appStorage1 = new VersionedLevelDbStorageAdapter(storage1);
appStorage2 = new VersionedLevelDbStorageAdapter(storage2);
}

public Logger logger = LogManager.getLogger(com.horizen.examples.DefaultApplicationState.class);

@Override
public void validate(SidechainStateReader stateReader, SidechainBlock block) throws IllegalArgumentException {
// do nothing always successful
Expand All @@ -24,11 +46,49 @@ public void validate(SidechainStateReader stateReader, BoxTransaction<Propositio

@Override
public Try<ApplicationState> onApplyChanges(SidechainStateReader stateReader, byte[] blockId, List<Box<Proposition>> newBoxes, List<byte[]> boxIdsToRemove) {
ByteArrayWrapper version = new ByteArrayWrapper(blockId);

ByteArrayWrapper key = new ByteArrayWrapper(DUMMY_KEY);
ByteArrayWrapper value = new ByteArrayWrapper(blockId);

List<Pair<ByteArrayWrapper, ByteArrayWrapper>> toUpdate = Arrays.asList(new Pair<>(key, value));

appStorage1.update(version, toUpdate, new ArrayList<>());
appStorage2.update(version, toUpdate, new ArrayList<>());

logger.info("Sidechain application state updated with version: " + BytesUtils.toHexString(version.data()));

return new Success<>(this);
}

@Override
public Try<ApplicationState> onRollback(byte[] blockId) {

appStorage1.rollback(new ByteArrayWrapper(blockId));
appStorage2.rollback(new ByteArrayWrapper(blockId));
logger.info(String.format("Application state has been rollback to version: '%s'", BytesUtils.toHexString(blockId)));

return new Success<>(this);
}

@Override
public boolean checkStoragesVersion(byte[] blockId)
{
byte[] ver1 = appStorage1.lastVersionID().orElse(new ByteArrayWrapper(NULL_VERSION)).data();
byte[] ver2 = appStorage2.lastVersionID().orElse(new ByteArrayWrapper(NULL_VERSION)).data();

logger.debug(String.format("Reference version: '%s', application state current versions: '%s', '%s'",
BytesUtils.toHexString(blockId),
BytesUtils.toHexString(ver1),
BytesUtils.toHexString(ver2))
);

return Arrays.equals(blockId, ver1) && Arrays.equals(blockId, ver2);
}

public void closeStorages() {
logger.debug("Closing storages");
appStorage1.close();
appStorage2.close();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,34 @@
import com.horizen.box.Box;
import com.horizen.proposition.Proposition;
import com.horizen.secret.Secret;
import com.horizen.storage.leveldb.VersionedLevelDbStorageAdapter;
import com.horizen.utils.ByteArrayWrapper;
import com.horizen.utils.BytesUtils;
import com.horizen.utils.Pair;
import com.horizen.wallet.ApplicationWallet;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

import java.io.File;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

public class DefaultApplicationWallet implements ApplicationWallet {

// two distinct storages are used in order to test a version misalignment during startup and the recover logic
private VersionedLevelDbStorageAdapter walletStorage1;
private VersionedLevelDbStorageAdapter walletStorage2;

public static byte[] NULL_VERSION = new byte[32];
public static byte[] DUMMY_KEY = BytesUtils.fromHexString("abcd");

public DefaultApplicationWallet(File storage1, File storage2) {
walletStorage1 = new VersionedLevelDbStorageAdapter(storage1);
walletStorage2 = new VersionedLevelDbStorageAdapter(storage2);
}

public Logger logger = LogManager.getLogger(com.horizen.examples.DefaultApplicationWallet.class);
@Override
public void onAddSecret(Secret secret) {

Expand All @@ -21,11 +43,45 @@ public void onRemoveSecret(Proposition proposition) {

@Override
public void onChangeBoxes(byte[] blockId, List<Box<Proposition>> boxesToUpdate, List<byte[]> boxIdsToRemove) {
ByteArrayWrapper version = new ByteArrayWrapper(blockId);

ByteArrayWrapper key = new ByteArrayWrapper(DUMMY_KEY);
ByteArrayWrapper value = new ByteArrayWrapper(blockId);

List<Pair<ByteArrayWrapper, ByteArrayWrapper>> toUpdate = Arrays.asList(new Pair<>(key, value));

walletStorage1.update(version, toUpdate, new ArrayList<>());
walletStorage2.update(version, toUpdate, new ArrayList<>());

logger.info("Sidechain application wallet updated with version: " + BytesUtils.toHexString(version.data()));
}

@Override
public void onRollback(byte[] blockId) {

walletStorage1.rollback(new ByteArrayWrapper(blockId));
walletStorage2.rollback(new ByteArrayWrapper(blockId));
logger.info(String.format("Application wallet has been rollback to version: '%s'", BytesUtils.toHexString(blockId)));
}

@Override
public boolean checkStoragesVersion(byte[] blockId)
{
byte[] ver1 = walletStorage1.lastVersionID().orElse(new ByteArrayWrapper(NULL_VERSION)).data();
byte[] ver2 = walletStorage2.lastVersionID().orElse(new ByteArrayWrapper(NULL_VERSION)).data();

logger.debug(String.format("Reference version: '%s', application wallet current versions: '%s', '%s'",
BytesUtils.toHexString(blockId),
BytesUtils.toHexString(ver1),
BytesUtils.toHexString(ver2))
);

return Arrays.equals(blockId, ver1) && Arrays.equals(blockId, ver2);
}

public void closeStorages() {
logger.debug("Closing storages");
walletStorage1.close();
walletStorage2.close();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import com.google.inject.name.Names;

import com.horizen.SidechainAppModule;
import com.horizen.SidechainAppStopper;
import com.horizen.SidechainSettings;
import com.horizen.api.http.ApplicationApiGroup;
import com.horizen.box.*;
Expand Down Expand Up @@ -42,10 +43,18 @@ public void configureApp() {
HashMap<Byte, SecretSerializer<Secret>> customSecretSerializers = new HashMap<>();
HashMap<Byte, TransactionSerializer<BoxTransaction<Proposition, Box<Proposition>>>> customTransactionSerializers = new HashMap<>();

ApplicationWallet defaultApplicationWallet = new DefaultApplicationWallet();
ApplicationState defaultApplicationState = new DefaultApplicationState();

String dataDirAbsolutePath = sidechainSettings.scorexSettings().dataDir().getAbsolutePath();

// two distinct storages are used in application state and wallet in order to test a version
// misalignment during startup and the recover logic
File appWalletStorage1 = new File(dataDirAbsolutePath + "/appWallet1");
File appWalletStorage2 = new File(dataDirAbsolutePath + "/appWallet2");
DefaultApplicationWallet defaultApplicationWallet = new DefaultApplicationWallet(appWalletStorage1, appWalletStorage2);

File appStateStorage1 = new File(dataDirAbsolutePath + "/appState1");
File appStateStorage2 = new File(dataDirAbsolutePath + "/appState2");
DefaultApplicationState defaultApplicationState = new DefaultApplicationState(appStateStorage1, appStateStorage2);

File secretStore = new File(dataDirAbsolutePath + "/secret");
File walletBoxStore = new File(dataDirAbsolutePath + "/wallet");
File walletTransactionStore = new File(dataDirAbsolutePath + "/walletTransaction");
Expand All @@ -67,7 +76,9 @@ public void configureApp() {
// For example new Pair("wallet, "allBoxes");
List<Pair<String, String>> rejectedApiPaths = new ArrayList<>();


// use a custom object which implements the stopAll() method
SidechainAppStopper applicationStopper = new SimpleAppStopper(
defaultApplicationState, defaultApplicationWallet);

bind(SidechainSettings.class)
.annotatedWith(Names.named("SidechainSettings"))
Expand Down Expand Up @@ -130,5 +141,9 @@ public void configureApp() {
bind(new TypeLiteral<List<Pair<String, String>>> () {})
.annotatedWith(Names.named("RejectedApiPaths"))
.toInstance(rejectedApiPaths);

bind(SidechainAppStopper.class)
.annotatedWith(Names.named("ApplicationStopper"))
.toInstance(applicationStopper);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
package com.horizen.examples;

import com.horizen.SidechainAppStopper;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

class SimpleAppStopper implements SidechainAppStopper {

public Logger logger = LogManager.getLogger(SimpleAppStopper.class);
private DefaultApplicationState appState;
private DefaultApplicationWallet appWallet;

SimpleAppStopper(DefaultApplicationState appState, DefaultApplicationWallet appWallet) { // pass storages themselves
this.appState = appState;
this.appWallet = appWallet;
}

@Override
public void stopAll() {
logger.info("...stopping application...");
appWallet.closeStorages();
appState.closeStorages();
}
}
3 changes: 2 additions & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>com.horizen</groupId>
<artifactId>Sidechains</artifactId>
<version>0.3.2</version>
<version>0.3.3</version>
<inceptionYear>2018</inceptionYear>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
Expand All @@ -17,5 +17,6 @@
<module>sdk</module>
<module>examples/simpleapp</module>
<module>tools/sctool</module>
<module>tools/dbtool</module>
</modules>
</project>
2 changes: 1 addition & 1 deletion qa/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ In order to run a python test for debugging SDK application, the following proce

1) When starting a sc node in the py test, add the option '_-agentlib_' to the _extra_args_ list in the relevant API call, for example:
```
start_sc_nodes(1, self.options.tmpdir, extra_args=[['-agentlib']])
start_sc_nodes(1, self.options.tmpdir, extra_args=['-agentlib'])
```
This will cause the simpleApp process to start with the debug agent acting as a server. The process will wait until the debugger has been connected.

Expand Down
2 changes: 1 addition & 1 deletion qa/SidechainTestFramework/sc_test_framework.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ def main(self):
help="Don't stop bitcoinds after the test execution")
parser.add_option("--zendir", dest="zendir", default="ZenCore/src",
help="Source directory containing zend/zen-cli (default: %default)")
parser.add_option("--scjarpath", dest="scjarpath", default="../examples/simpleapp/target/sidechains-sdk-simpleapp-0.3.2.jar;../examples/simpleapp/target/lib/* com.horizen.examples.SimpleApp", #New option. Main class path won't be needed in future
parser.add_option("--scjarpath", dest="scjarpath", default="../examples/simpleapp/target/sidechains-sdk-simpleapp-0.3.3.jar;../examples/simpleapp/target/lib/* com.horizen.examples.SimpleApp", #New option. Main class path won't be needed in future
help="Directory containing .jar file for SC (default: %default)")
parser.add_option("--tmpdir", dest="tmpdir", default=tempfile.mkdtemp(prefix="sc_test"),
help="Root directory for datadirs")
Expand Down
Loading

0 comments on commit ae1b1ab

Please sign in to comment.