Provide example on how to upload custom modpack/datapack/world files

[garrettjoecox]

I'm trying to get this set up for various worlds/modpacks, and it's not clear to me how to use an existing world, or upload a modpack zip for minecraft-docker to use, or add datapacks to an existing world

[jr298]

Or even an additional container, or EC2 instance to be manually started that has the EFS mounted.
AWS seems to take pleasure in being perverse... I can create an EC2 instance which has the EFS mounted, but can't ssh to it, or I can create an instance that can't mount the EFS, but that I can ssh to...

Brilliant deployment - insanely frustrating to actually configure the game at all.

[doctorray117]

@garrettjoecox You'd want to look at https://github.com/itzg/docker-minecraft-server/blob/master/README.md and scroll down to "Running a Forge Server" or equivalent for the mod you want to use. I experimented with this and found that it's nothing short of a pain in the ass. Many mods are tied to specific versions of minecraft/forge/paper/bukkit/whatever and finding the right cocktail of software can be difficult.

My recommendation would be to try and get something working locally with the minecraft docker image that creates a world, starts up properly, and lets you connect to it on your computer, and then you can translate those startup commands/variables to this implementation if desired. I'd start with a clean /data folder each time (or just don't mount one, let it run internally) because every time you mess with an option it's going to potentially conflict with existing files.

[garrettjoecox]

Yeah, I played around with those configuration options but those only seem to account for a fresh server, what if I have an existing world, homemade modpack, or datapacks I want to add to the file system? How can I gain access to the file system is probably a clearer way of putting my question.

[doctorray117]

Ah, that I can help with. Start here: https://github.com/doctorray117/minecraft-ondemand#usage-and-customization

Short answer, one of two options:

1. Mount the EFS to an compatible EC2 instance (linux only) and copy your files in/out from there
2. Use Datasync to make a bidirectional copy function to an S3 bucket for easier upload/download access. The main difference here is that you want to copy everything instead of just the config files, so don't specify any includes/excludes.

[garrettjoecox]

I'd prefer to go with option 1, as that sounds easier, but I'm afraid I'm running into what @jr298 mentioned. This is likely due to unfamiliarity with AWS networking stuff.. If I boot up an EC2 instance I can ssh into it fine, but if I add it to the minecraft VPC(which seems to be required to mount the EFS) and mount the EFS I can no longer SSH into it.

[garrettjoecox]

In the mean time I'll start making attempts towards option 2!

[doctorray117]

For #1, you'd launch an instance in one of the default vpc subnets with "assign public IP" set to enabled. The security group would need to allow you to ssh in, and to reach efs you'd probably have to create an role for the instance and assign the efs policy to it, and follow the mounting instruction shown in the efs console. Easy right? Just kidding.

Maybe I'll make a tutorial/quick-start for that. I was also thinking a simple preconfigured container with an sftp relay might be appropriate for this application.

[jr298]

"Easy right? Just kidding."

I ended up removing the security groups and mount group/point? then setting up a group with completely open networking to get into it. Then putting it back as it was.

Took far more hours than it should have done (I'm reasonably adept with normal networking/firewalls etc, but AWS is currently a black box in a coal cellar at midnight... and the documentation is like a broken torch.

An sftp relay server would be great - simple access for anyone.
Happy to test it as an AWS ignoramus

[garrettjoecox]

I finally managed to get something working, not sure if I broke something in the process, I had to go into EFS > minecraft-server-stack/FileSystem > Network tab, remove one of the availability zones (all three are on the isolated subnets) and re-add but with a public subnet selected.

I was then able to launch an instance that I could ssh into, that also had the EFS mounted at /mnt/efs/fs1

[garrettjoecox]

I'll have to try when I get home but I think this could be automated with

    const fileSystem = new efs.FileSystem(this, 'FileSystem', {
      vpc,
      removalPolicy: RemovalPolicy.SNAPSHOT,
+     vpcSubnets: {
+       subnetType: ec2.SubnetType.PUBLIC
+     }
    });

At

minecraft-ondemand/cdk/lib/minecraft-stack.ts

Line 38 in 6b2d1c4

const fileSystem = new efs.FileSystem(this, 'FileSystem', {

[doctorray117]

I'm messing with CDK this weekend to expand the documentation so I'll try to replicate this a bit. Were you able to test your changes?

[AdamMarciniak]

I finally managed to get something working, not sure if I broke something in the process, I had to go into EFS > minecraft-server-stack/FileSystem > Network tab, remove one of the availability zones (all three are on the isolated subnets) and re-add but with a public subnet selected.

I was then able to launch an instance that I could ssh into, that also had the EFS mounted at /mnt/efs/fs1

This worked for me! Thank you!

[Stealthii]

@garrettjoecox as an example of how I manage my own mods - using a public S3 bucket (you can use Cloudfront if you want), and making these options available to your container via URL is the way to go. The advantage of this is you can define your config and your mods outside of the container storage (and blow it away each time if you want), concerning yourself only with world storage persistance.

As an example of running a Fabric 1.18.x server with a packwiz modpack, some additional mods not on CurseForge/Modrinth, and custom configuration (inc. whitelist), with the following env vars for the server container:

  "environment": [
        {
          "name": "ENFORCE_WHITELIST",
          "value": "TRUE"
        },
        {
          "name": "MODCONFIG",
          "value": "https://pepsi.dog/minecraft/files/plscraft-s3_config_v4.zip"
        },
        {
          "name": "MODS",
          "value": "https://pepsi.dog/minecraft/jar/OpenLoader-fabric-1.18.1-3.0.0.jar,https://ci.opencollab.dev/job/GeyserMC/job/Geyser-Fabric/job/java-1.18/lastSuccessfulBuild/artifact/build/libs/Geyser-Fabric-2.0.0-SNAPSHOT.jar"
        },
        {
          "name": "OVERRIDE_WHITELIST",
          "value": "TRUE"
        },
        {
          "name": "PACKWIZ_URL",
          "value": "https://pepsidogs.github.io/pepsipack/pack.toml"
        },
        {
          "name": "REMOVE_OLD_MODS",
          "value": "TRUE"
        },
        {
          "name": "TYPE",
          "value": "FABRIC"
        },
        {
          "name": "VERSION",
          "value": "1.18.1"
        },
        {
          "name": "WHITELIST_FILE",
          "value": "https://pepsi.dog/minecraft/whitelist.json"
        }
      ]

If you want to keep things even simpler, I'd suggest creating your own packwiz pack and hosting it for free via Github Pages - here's an example of my own pack, which contains both server and client side mods, and config (for both the server or client) can be laid down / updated by packwiz.

To update mods or config, I just push updates to that repo, and the next time the ECS container starts it'll pull down from that github.io link.

[yijiazz]

@Stealthii Thanks for sharing the approach! I like this the idea of managing mods outside of the container storage. I'm fairly new to MC servers, creating modpack and Github page, so have a few questions, hope you don't mind :)

1. In what file are you storing your environment json? My impression is that server configs can be provided in either server.properties or as an docker run -e, but not in json format. Or, was it just an output from the ECS task?

2. When server sees <host>/pack.toml during first start, does it download the jars and configs to the file system? Does it re-download in subsequent downloads?

3. Why do you also include client side mods in your packwiz pack? When server starts and it sees client packs, does it actually do anything (like forcing each client to download the pack)?

Thanks!

[Stealthii]

@yijiazz no problem! Good questions:

1. Those are the environment settings from the ECS task. As per itzg's docs, you can override server.properties values on each run.
2. Packwiz will download and verify the metadata, redownloading any metadata files that have changed, and subsequently downloading any files where the metadata doesn't match. Any files not defined in the pack are untouched, so if you have extra mods defined with MODS for example, packwiz ignores these (unless the naming conflicts).
3. Pepsipack is a light modpack for my friends and our server. It's honestly trivial to have client/server/both defined in the one pack, and honestly desirable. This means recommended or required mods can be defined for clients to connect (assuming they are using packwiz. It also means if I update mods (such as Simple Voice Chat), both the server, and connecting clients pull that update on next start (as some versions implement protocol changes requiring an update).

[Rithmatist]

I am hosting tekkit2 modpack serverless and It's actually pretty simple to do it, and the best part is that you don't even have to use another docker image. Just read the documentation on his Github.

[gwwatkin]

Hi, of the 3 custom uploads I'm having trouble with world files. I was able to upload world files with the s3/data transfer methods, but then the server crashes with this permission error on startup:

java.nio.file.AccessDeniedException: ./world/session.lock

I'm guessing the permissions get overwritten by the sync and the server somehow loses write permissions.

Been trying to access the EFS volume by mounting with EC2 as suggested here but still can't get SSH to the machine in the VPC going, even with the public IP set

[gwwatkin]

Following this approach of moving an endpoint to a public subnet worked for me and I was able to SSH into an ec2 with access to the EFS.

There I could indeed confirm that the permissions where broken by the sync:

drwxrwxr-x 2 ec2-user  ec2-user   6144 Jan  4 18:54 advancements
drwxrwxr-x 2 ec2-user  ec2-user   6144 Jan  4 18:54 data
drwxrwxr-x 2 ec2-user  ec2-user   6144 Jan  4 18:47 datapacks
drwxrwxr-x 6 ec2-user  ec2-user   6144 Jan  4 18:48 DIM1
drwxrwxr-x 6 ec2-user  ec2-user   6144 Jan  4 18:48 DIM-1
drwxrwxr-x 2 ec2-user  ec2-user  14336 Jan  4 18:49 entities
drwxr-xr-x 2 nfsnobody nfsnobody  6144 Jan  1  1970 gc-logs
-rwxr-xr-x 1 nfsnobody nfsnobody  1474 Jan  5 00:28 level.dat
-rwxr-xr-x 1 nfsnobody nfsnobody  1471 Jan  5 00:28 level.dat_old
drwxrwxr-x 2 ec2-user  ec2-user   6144 Jan  4 19:37 playerdata
drwxrwxr-x 2 ec2-user  ec2-user  14336 Jan  4 19:13 poi
-rwxr-xr-x 1 nfsnobody nfsnobody  1634 Jan  5 00:28 realms-upload.log
drwxrwxr-x 2 ec2-user  ec2-user  22528 Jan  4 18:56 region
-rwxr-xr-x 1 nfsnobody nfsnobody     3 Jan  5 00:32 session.lock
drwxrwxr-x 2 ec2-user  ec2-user   6144 Jan  4 18:54 stats

Fixed by sudo chown -R ec2-user . && sudo chgrp -R ec2-user .

Server works! I'm very happy 😁

[MarkKoz]

Several pointers on mounting EFS:

1. Make sure you create your EC2 instance in the same VPC — by default the minecraft stack creates a new VPC.
2. In EFS, delete the mount target in the same AZ as your EC2 instance, and re-add it using a public subnet (as suggested in an earlier comment).
3. Make sure the security group for the mount target has an NFS inbound rule for the EC2 instance's security group (see this).
4. Use the Amazon Linux AMI as it seems to be the most convenient way to install the NFS dependencies.