feat: Extract registries data from server.jar (#58)

To allow players to place arbitrary blocks and use any of the items the
game has to offer, we need access to a list of these things. Fortunately,
this data is available from the official Minecraft server .jar file via an
integrated command-line interface.

This patch adds a Makefile target that downloads the .jar and
extracts relevant data from it, resulting in multiple JSON files.
Furthermore, this patch adds a handmade JSON parser that is used
to parse the "registries.json" file at run-time. The JSON parser is
covered by unit tests.

The registries data is not yet widely used, but as a proof-of-concept,
the protocol ID of the player entity type is now extracted from it
instead of being hardcoded. This, along with future patches, will
greatly reduce the amount of work needed to support additional
features and perform version upgrades.

Finally, this patch overhauls the Dockerfile such that the data
extraction can be performed at build time, while keeping a slim final
image with only the relevant parts.

.gitignore

@@ -1,5 +1,6 @@
 .idea
 *.so
 /cobolcraft
+/data
 /test
 /test.log

Dockerfile

@@ -1,8 +1,11 @@
-FROM debian:bookworm-slim
+# --- Build stage ---
+FROM debian:bookworm-slim AS build
 
-# Install dependencies
+# Install packages required for building
 ENV DEBIAN_FRONTEND=noninteractive
-RUN apt-get update && apt-get install -y tini gcc g++ make gnucobol && rm -rf /var/lib/apt/lists/*
+RUN apt-get update && \
+    apt-get install -y gcc g++ make gnucobol curl default-jre-headless && \
+    rm -rf /var/lib/apt/lists/*
 
 # Copy source files
 COPY Makefile .
@@ -15,6 +18,22 @@ COPY blobs ./blobs
 # Build
 RUN make
 
+# --- Runtime stage ---
+FROM debian:bookworm-slim
+
+# Install runtime packages
+ENV DEBIAN_FRONTEND=noninteractive
+RUN apt-get update && \
+    apt-get install -y gnucobol tini && \
+    rm -rf /var/lib/apt/lists/*
+
+# Copy the build results
+COPY --from=build Makefile .
+COPY --from=build cobolcraft .
+COPY --from=build *.so .
+COPY --from=build blobs ./blobs
+COPY --from=build data/generated/reports/*.json ./data/generated/reports/
+
 # Include runtime dependencies
 ENV COB_PRE_LOAD=CBL_GC_SOCKET:COBOLCRAFT_UTIL
 

Makefile

@@ -1,14 +1,22 @@
 # Compiler
 COBC = cobc
 
+# Libraries
 SOCKET_LIB = CBL_GC_SOCKET.so
 UTIL_LIB = COBOLCRAFT_UTIL.so
+
+# Sources and binary
 SRC = main.cob src/*.cob src/*/*.cob
 BIN = cobolcraft
+
+# Data extraction from Mojang's server.jar
+JSON_DATA = data/generated/reports/registries.json data/generated/reports/blocks.json
+
+# Test sources and binary
 TEST_SRC = test.cob tests/*.cob
 TEST_BIN = test
 
-all: $(BIN)
+all: $(BIN) $(JSON_DATA)
 
 $(SOCKET_LIB):
 	cd CBL_GC_SOCKET && ./build.sh
@@ -17,16 +25,22 @@ $(SOCKET_LIB):
 $(UTIL_LIB): cpp/cobolcraft_util.cpp
 	g++ -shared -Wall -O2 -fPIC -o $@ $<
 
+$(JSON_DATA):
+	mkdir -p data
+	curl -o data/server.jar https://piston-data.mojang.com/v1/objects/8dd1a28015f51b1803213892b50b7b4fc76e594d/server.jar
+	cd data && java -DbundlerMainClass="net.minecraft.data.Main" -jar server.jar --reports
+
 $(BIN): $(SOCKET_LIB) $(UTIL_LIB) $(SRC)
 	$(COBC) -x -debug -Wall -fnotrunc --free -lstdc++ -o $@ $(SRC)
 
 clean:
-	rm -f $(BIN)
-	rm -f $(SOCKET_LIB)
-	rm -f $(UTIL_LIB)
-	rm -f $(TEST_BIN)
+	rm $(BIN)
+	rm $(SOCKET_LIB)
+	rm $(UTIL_LIB)
+	rm $(TEST_BIN)
+	rm -r data/
 
-run: $(BIN)
+run: $(BIN) $(JSON_DATA)
 	COB_PRE_LOAD=CBL_GC_SOCKET:COBOLCRAFT_UTIL ./$(BIN)
 
 $(TEST_BIN): $(TEST_SRC) $(SRC) $(UTIL_LIB)

README.md

@@ -28,6 +28,8 @@ To deploy on Linux, make sure all prerequisites are installed:
 * `cobc` (e.g. from the `gnucobol` APT package on Debian)
 * `make`
 * `g++`
+* `curl` (needed to download the official server .jar)
+* a recent version of Java (needed to extract data from the server .jar)
 
 Then execute `make` to build, followed by `make run` to start a server on port 25565.
 
@@ -58,7 +60,7 @@ Writing a Minecraft server was perhaps not the best idea for a first COBOL proje
 no functionality regarding low-level data manipulation (bits and bytes) which the Minecraft protocol needs lots of.
 But remember: I didn't know this starting out, and quitting before having a working prototype was not on the table!
 A lot of this functionality had to be implemented completely from scratch.
-For large, complex data, I opted for recording packets from a "real" server via Wireshark (see `blobs` directory) and
+For large, complex data, I opted for recording packets from a "real" server via Wireshark (see `blobs/` directory) and
 playing them back to clients.
 
 If you too have never written COBOL before but are interested in CobolCraft, I recommend reading the GnuCOBOL
@@ -68,6 +70,40 @@ https://gnucobol.sourceforge.io/HTML/gnucobpg.html
 To learn more about the Minecraft protocol, you can refer to https://wiki.vg/Protocol.
 In some cases, it may be helpful to look at real server traffic to better understand the flow of information.
 
+## Program Overview
+
+This section provides a high-level overview of CobolCraft from a software design viewpoint.
+
+### Source Components
+
+The program entrypoint is `main.cob`.
+The remaining COBOL sources are located in the `src/` directory, including `src/server.cob`, which contains the bulk
+of CobolCraft.
+
+Some functions had to be implemented in C++, such as process signal handling, retrieving accurate timestamps, or
+conversion of IEEE-754 floating-point numbers.
+These sources are located in the `cpp/` directory and get compiled into a shared library (`.so` on Linux).
+
+TCP sockets are managed by the CBL_GC_SOCKET socket library located in the `CBL_GC_SOCKET/` directory.
+
+### Packet Blobs
+
+CobolCraft makes use of network data captured from an instance of the official server application.
+This data is located in the `blobs/` directory and is decoded at run-time.
+
+### Data Extraction
+
+The official Minecraft (Java Edition) server and client applications contain large amounts of data such as:
+
+* block and item types
+* entity types
+* biomes
+
+Fortunately, the freely available server .jar offers a command-line interface for extracting this data as JSON.
+The CobolCraft `Makefile` has a target that downloads the .jar and extracts the JSON data from it.
+The JSON files are evaluated at runtime using a custom-built generic JSON parser, such that CobolCraft can
+inter-operate successfully with the Minecraft client without distributing potentially copyrighted material.
+
 ## Legal Notices
 
 This project (except 3rd-party contents as stated below) is licensed under the MIT License.
@@ -79,3 +115,4 @@ Note that line 939 of `CBL_GC_SOCKET/cob_socket.cpp` has been modified (compared
 library) to fix improper `snprintf` usage.
 
 "Minecraft" is a trademark of Mojang Synergies AB.
+CobolCraft is neither affiliated with nor endorsed by Mojang.