[WIP] textual chat format (#1144)

[MiniDigger]

Implements (#1144)

for examples of the format, check the tests.

feedback welcome!

TODO:

• we need a way to convert enhanced format strings -> bungee chat components
• do we need a way to convert bungee chat components -> enhanced format strings too?
• we need a way to escape tokens (player input)
• we need a way to strip tokens (ChatColor.stripColors, maybe bungees toPlainText can be of use)
• we need a setEnhancedFormat method in ChatEvents (with placeholders for name and msg)
• we need getters and setters that use the enhanced string format for chat events
• we need a sendMessage method that uses the enhanced string format for players (what about console?)
• we need a config option to hide the stuff behind (what exactly should it do? how should the api methods behave when the option is false?)
• check that BufferedCommandSender works as expected
• allow CraftBlockCommandSender, RemoteCommandSender to send actual components, not just plaintext

Format Todos (not sure if I wan to do all of them quite yet):

• colors
• decoration
• text
• click
• hover
• insertion
• translation
• keybinding
• selector
• score

[MiniDigger]

Now with a actually clean branch.
I also need feedback on how the Bungee Chat API -> Textual Chat String conversion should look like:
https://github.com/PaperMC/Paper/pull/1497/files#diff-3a90edf0dc77f49bb788499ba586bde9R491

[MiniDigger]

rewrote the tokenizer, it now uses regex. makes it easier to implement the strip and escape methods.
since I am at work and I can't update the patch, ill attach the two classes here for now

click me

package test;

import java.util.EnumSet;
import java.util.Map;
import java.util.Optional;
import java.util.Stack;
import java.util.function.Consumer;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import javax.annotation.Nonnull;

import net.md_5.bungee.api.ChatColor;
import net.md_5.bungee.api.chat.BaseComponent;
import net.md_5.bungee.api.chat.ClickEvent;
import net.md_5.bungee.api.chat.ComponentBuilder;
import net.md_5.bungee.api.chat.HoverEvent;

public class ChatParser {

	private static Pattern pattern = Pattern
			.compile("((?<start>\\{)(?<token>([^{}]+)|([^{}]+\"(?<inner>[^\"]+)\"))(?<end>\\}))+?");

	@Nonnull
	public static String escapeTokens(@Nonnull String richMessage) {
		StringBuilder sb = new StringBuilder();
		Matcher matcher = pattern.matcher(richMessage);
		int lastEnd = 0;
		while (matcher.find()) {
			int startIndex = matcher.start();
			int endIndex = matcher.end();

			if (startIndex > lastEnd) {
				sb.append(richMessage.substring(lastEnd, startIndex));
			}
			lastEnd = endIndex;

			String start = matcher.group("start");
			String token = matcher.group("token");
			String inner = matcher.group("inner");
			String end = matcher.group("end");

			// also escape inner
			if (inner != null) {
				token = token.replace(inner, escapeTokens(inner));
			}

			sb.append("\\").append(start).append(token).append("\\").append(end);
		}

		if (richMessage.length() > lastEnd) {
			sb.append(richMessage.substring(lastEnd, richMessage.length()));
		}

		return sb.toString();
	}

	@Nonnull
	public static String stripTokens(@Nonnull String richMessage) {
		StringBuilder sb = new StringBuilder();
		Matcher matcher = pattern.matcher(richMessage);
		int lastEnd = 0;
		while (matcher.find()) {
			int startIndex = matcher.start();
			int endIndex = matcher.end();

			if (startIndex > lastEnd) {
				sb.append(richMessage.substring(lastEnd, startIndex));
			}
			lastEnd = endIndex;
		}

		if (richMessage.length() > lastEnd) {
			sb.append(richMessage.substring(lastEnd, richMessage.length()));
		}

		return sb.toString();
	}

	@Nonnull
	public static String handlePlaceholders(@Nonnull String richMessage, @Nonnull String... placeholders) {
		if (placeholders.length % 2 != 0) {
			throw new RuntimeException(
					"Invalid number placeholders defined, usage: parseFormat(format, key, value, key, value...)");
		}
		for (int i = 0; i < placeholders.length; i += 2) {
			richMessage = richMessage.replace("{" + placeholders[i] + "}", placeholders[i + 1]);
		}
		return richMessage;
	}

	@Nonnull
	public static String handlePlaceholders(@Nonnull String richMessage, @Nonnull Map<String, String> placeholders) {
		for (Map.Entry<String, String> entry : placeholders.entrySet()) {
			richMessage = richMessage.replace(entry.getKey(), entry.getValue());
		}
		return richMessage;
	}

	@Nonnull
	public static BaseComponent[] parseFormat(@Nonnull String richMessage, @Nonnull String... placeholders) {
		return parseFormat(handlePlaceholders(richMessage, placeholders));
	}

	@Nonnull
	public static BaseComponent[] parseFormat(@Nonnull String richMessage, @Nonnull Map<String, String> placeholders) {
		return parseFormat(handlePlaceholders(richMessage, placeholders));
	}

	@Nonnull
	public static BaseComponent[] parseFormat(@Nonnull String richMessage) {
		ComponentBuilder builder = null;

		Stack<ClickEvent> clickEvents = new Stack<>();
		Stack<HoverEvent> hoverEvents = new Stack<>();
		Stack<ChatColor> colors = new Stack<>();
		EnumSet<TextDecoration> decorations = EnumSet.noneOf(TextDecoration.class);

		Matcher matcher = pattern.matcher(richMessage);
		int lastEnd = 0;
		while (matcher.find()) {
			int startIndex = matcher.start();
			int endIndex = matcher.end();

			String msg = null;
			if (startIndex > lastEnd) {
				msg = richMessage.substring(lastEnd, startIndex);
			}
			lastEnd = endIndex;

			// handle message
			if (msg != null && msg.length() != 0) {
				// append message
				if (builder == null) {
					builder = new ComponentBuilder(msg);
				} else {
					builder.append(msg, ComponentBuilder.FormatRetention.NONE);
				}

				// set everything that is not closed yet
				if (clickEvents.size() > 0) {
					builder.event(clickEvents.peek());
				}
				if (hoverEvents.size() > 0) {
					builder.event(hoverEvents.peek());
				}
				if (colors.size() > 0) {
					builder.color(colors.peek());
				}
				if (decorations.size() > 0) {
					// no lambda because builder isn't effective final :/
					for (TextDecoration decor : decorations) {
						decor.apply(builder);
					}
				}
			}

			String group = matcher.group();
			String start = matcher.group("start");
			String token = matcher.group("token");
			String inner = matcher.group("inner");
			String end = matcher.group("end");

			Optional<TextDecoration> deco;
			Optional<ChatColor> color;

			// click
			if (token.startsWith("click:")) {
				clickEvents.push(handleClick(token, inner));
			} else if (token.equals("/click")) {
				clickEvents.pop();
			}
			// hover
			else if (token.startsWith("hover:")) {
				hoverEvents.push(handleHover(token, inner));
			} else if (token.equals("/hover")) {
				hoverEvents.pop();
			}
			// decoration
			else if ((deco = resolveDecoration(token)).isPresent()) {
				decorations.add(deco.get());
			} else if (token.startsWith("/") && (deco = resolveDecoration(token.replace("/", ""))).isPresent()) {
				decorations.remove(deco.get());
			}
			// color
			else if ((color = resolveColor(token)).isPresent()) {
				colors.push(color.get());
			} else if (token.startsWith("/") && resolveColor(token.replace("/", "")).isPresent()) {
				colors.pop();
			}
		}

		// handle last message part
		if (richMessage.length() > lastEnd) {
			String msg = richMessage.substring(lastEnd, richMessage.length());
			// append message
			if (builder == null) {
				builder = new ComponentBuilder(msg);
			} else {
				builder.append(msg, ComponentBuilder.FormatRetention.NONE);
			}

			// set everything that is not closed yet
			if (clickEvents.size() > 0) {
				builder.event(clickEvents.peek());
			}
			if (hoverEvents.size() > 0) {
				builder.event(hoverEvents.peek());
			}
			if (colors.size() > 0) {
				builder.color(colors.peek());
			}
			if (decorations.size() > 0) {
				// no lambda because builder isn't effective final :/
				for (TextDecoration decor : decorations) {
					decor.apply(builder);
				}
			}
		}

		if (builder == null) {
			// lets just return an empty component
			builder = new ComponentBuilder("");
		}

		return builder.create();
	}

	@Nonnull
	private static ClickEvent handleClick(@Nonnull String token, @Nonnull String inner) {
		String[] args = token.split(":");
		ClickEvent clickEvent;
		if (args.length < 2) {
			throw new RuntimeException("Can't parse click action (too few args) " + token);
		}
		switch (args[1]) {
		case "run_command":
			clickEvent = new ClickEvent(ClickEvent.Action.RUN_COMMAND, token.replace("click:run_command:", ""));
			break;
		case "suggest_command":
			clickEvent = new ClickEvent(ClickEvent.Action.SUGGEST_COMMAND, token.replace("click:suggest_command:", ""));
			break;
		case "open_url":
			clickEvent = new ClickEvent(ClickEvent.Action.OPEN_URL, token.replace("click:open_url:", ""));
			break;
		case "change_page":
			clickEvent = new ClickEvent(ClickEvent.Action.CHANGE_PAGE, token.replace("click:change_page:", ""));
			break;
		default:
			throw new RuntimeException("Can't parse click action (invalid type " + args[1] + ") " + token);
		}
		return clickEvent;
	}

	@Nonnull
	private static HoverEvent handleHover(@Nonnull String token, @Nonnull String inner) {
		String[] args = token.split(":");
		HoverEvent hoverEvent;
		if (args.length < 2) {
			throw new RuntimeException("Can't parse hover action (too few args) " + token);
		}
		switch (args[1]) {
		case "show_text":
			hoverEvent = new HoverEvent(HoverEvent.Action.SHOW_TEXT, parseFormat(inner));
			break;
		case "show_item":
			hoverEvent = new HoverEvent(HoverEvent.Action.SHOW_ITEM, parseFormat(inner));
			break;
		case "show_entity":
			hoverEvent = new HoverEvent(HoverEvent.Action.SHOW_ENTITY, parseFormat(inner));
			break;
		default:
			throw new RuntimeException("Can't parse hover action (invalid type " + args[1] + ") " + token);
		}
		return hoverEvent;
	}

	@Nonnull
	private static Optional<ChatColor> resolveColor(@Nonnull String token) {
		try {
			return Optional.of(ChatColor.valueOf(token.toUpperCase()));
		} catch (IllegalArgumentException ex) {
			return Optional.empty();
		}
	}

	@Nonnull
	private static Optional<TextDecoration> resolveDecoration(@Nonnull String token) {
		try {
			return Optional.of(TextDecoration.valueOf(token.toUpperCase()));
		} catch (IllegalArgumentException ex) {
			return Optional.empty();
		}
	}

	enum TextDecoration {
		BOLD(builder -> builder.bold(true)), //
		ITALIC(builder -> builder.italic(true)), //
		UNDERLINE(builder -> builder.underlined(true)), //
		STRIKETHROUGH(builder -> builder.strikethrough(true)), //
		OBFUSCATED(builder -> builder.obfuscated(true));

		private Consumer<ComponentBuilder> builder;

		TextDecoration(Consumer<ComponentBuilder> builder) {
			this.builder = builder;
		}

		public void apply(ComponentBuilder comp) {
			builder.accept(comp);
		}
	}
}

package test;

import net.md_5.bungee.api.chat.BaseComponent;
import net.md_5.bungee.chat.ComponentSerializer;

import org.junit.Test;

import static org.junit.Assert.assertEquals;

public class ChatParserTest {

	@Test
	public void testStripSimple() {
		String input = "{yellow}TEST{green} nested{/green}Test";
		String expected = "TEST nestedTest";
		assertEquals(expected, ChatParser.stripTokens(input));
	}

	@Test
	public void testStripComplex() {
		String input = "{yellow}{test} random {bold}stranger{/bold}{click:run_command:test command}{underline}{red}click here{/click}{blue} to {bold}FEEL{/underline} it";
		String expected = " random strangerclick here to FEEL it";
		assertEquals(expected, ChatParser.stripTokens(input));
	}

	@Test
	public void testStripInner() {
		String input = "{hover:show_text:\"{red}test:TEST\"}TEST";
		String expected = "TEST";
		assertEquals(expected, ChatParser.stripTokens(input));
	}
	
	@Test
	public void testEscapeSimple() {
		String input = "{yellow}TEST{green} nested{/green}Test";
		String expected = "\\{yellow\\}TEST\\{green\\} nested\\{/green\\}Test";
		assertEquals(expected, ChatParser.escapeTokens(input));
	}

	@Test
	public void testEscapeComplex() {
		String input = "{yellow}{test} random {bold}stranger{/bold}{click:run_command:test command}{underline}{red}click here{/click}{blue} to {bold}FEEL{/underline} it";
		String expected = "\\{yellow\\}\\{test\\} random \\{bold\\}stranger\\{/bold\\}\\{click:run_command:test command\\}\\{underline\\}\\{red\\}click here\\{/click\\}\\{blue\\} to \\{bold\\}FEEL\\{/underline\\} it";
		assertEquals(expected, ChatParser.escapeTokens(input));
	}

	@Test
	public void testEscapeInner() {
		String input = "{hover:show_text:\"{red}test:TEST\"}TEST";
		String expected = "\\{hover:show_text:\"\\{red\\}test:TEST\"\\}TEST";
		assertEquals(expected, ChatParser.escapeTokens(input));
	}


	@Test
	public void checkPlaceholder() {
		String input = "{test}";
		String expected = "{\"text\":\"Hello!\"}";
		BaseComponent[] comp = ChatParser.parseFormat(input, "test", "Hello!");

		test(comp, expected);
	}

	@Test
	public void testNiceMix() {
		String input = "{yellow}{test} random {bold}stranger{/bold}{click:run_command:test command}{underline}{red}click here{/click}{blue} to {bold}FEEL{/underline} it";
		String expected = "{\"extra\":[{\"color\":\"yellow\",\"text\":\"Hello! random \"},{\"color\":\"yellow\",\"bold\":true,\"text\":\"stranger\"},{\"color\":\"red\",\"underlined\":true,\"clickEvent\":{\"action\":\"run_command\",\"value\":\"test command\"},\"text\":\"click here\"},{\"color\":\"blue\",\"underlined\":true,\"text\":\" to \"},{\"color\":\"blue\",\"bold\":true,\"underlined\":true,\"text\":\"FEEL\"},{\"color\":\"blue\",\"bold\":true,\"text\":\" it\"}],\"text\":\"\"}";
		BaseComponent[] comp = ChatParser.parseFormat(input, "test", "Hello!");

		test(comp, expected);
	}

	@Test
	public void testColorSimple() {
		String input = "{yellow}TEST";
		String expected = "{\"color\":\"yellow\",\"text\":\"TEST\"}";

		test(input, expected);
	}

	@Test
	public void testColorNested() {
		String input = "{yellow}TEST{green}nested{/green}Test";
		String expected = "{\"extra\":[{\"color\":\"yellow\",\"text\":\"TEST\"},{\"color\":\"green\",\"text\":\"nested\"},{\"color\":\"yellow\",\"text\":\"Test\"}],\"text\":\"\"}";

		test(input, expected);
	}

	@Test
	public void testColorNotNested() {
		String input = "{yellow}TEST{/yellow}{green}nested{/green}Test";
		String expected = "{\"extra\":[{\"color\":\"yellow\",\"text\":\"TEST\"},{\"color\":\"green\",\"text\":\"nested\"},{\"text\":\"Test\"}],\"text\":\"\"}";

		test(input, expected);
	}

	@Test
	public void testHover() {
		String input = "{hover:show_text:\"{red}test\"}TEST";
		String expected = "{\"hoverEvent\":{\"action\":\"show_text\",\"value\":[{\"color\":\"red\",\"text\":\"test\"}]},\"text\":\"TEST\"}";

		test(input, expected);
	}

	@Test
	public void testHoverWithColon() {
		String input = "{hover:show_text:\"{red}test:TEST\"}TEST";
		String expected = "{\"hoverEvent\":{\"action\":\"show_text\",\"value\":[{\"color\":\"red\",\"text\":\"test:TEST\"}]},\"text\":\"TEST\"}";

		test(input, expected);
	}

	@Test
	public void testClick() {
		String input = "{click:run_command:test}TEST";
		String expected = "{\"clickEvent\":{\"action\":\"run_command\",\"value\":\"test\"},\"text\":\"TEST\"}";

		test(input, expected);
	}

	@Test
	public void testClickExtendedCommand() {
		String input = "{click:run_command:test command}TEST";
		String expected = "{\"clickEvent\":{\"action\":\"run_command\",\"value\":\"test command\"},\"text\":\"TEST\"}";

		test(input, expected);
	}

	private void test(String input, String expected) {
		test(ChatParser.parseFormat(input), expected);
	}

	private void test(BaseComponent[] comp, String expected) {
		assertEquals(expected, ComponentSerializer.toString(comp));
	}
}

[aikar]

regex is concerning for recursive depth, see:

{red}Hello: {yellow}{hover:show_text:\"{red}test:TEST{/red}\"}TEST{/hover}{/yellow}

I also don't think I like optional close tags.

[aikar]

also not really a fan of { }, it looks too much like JSON and is confusing to think "was this supposed to be a JSON format of the message? but it's not correct"

I would prefer [ ] as the token characters myself.

Maybe also use attribute style?

[red][hover type="text" text="[green]Hello[/green]"]Red Text with Hover[/hover][/red]

this is where I think regex will fail hardcore.

What if it then becomes

[red][hover type="text" text="[green]Hello[/green]"]Red Text with Hover[/hover] Red text without hover[/red]

Maintaining position in the state processing would be more difficult with regex.

We're also going to need place holder support with a vararg signature for items/entities:

player.sendRichMessage("[red][hover type=entity entity=%1]Hover for entity: %2[/hover][/red]", entity, entity.getName());

something like that?

[kashike]

or < and />, like HTML tags.

[MiniDigger]

@aikar nesting isn't a problem with my regex, it detects nested tags and will parse them recursively. If wanted, we can limit recursive depth (everything > 1 is not displayed by the vanilla client anyways)

I am really not a fan of attribute style messages. That would make them harder to write again as they are much more verbose.

Placeholder support is already build in currently, but not via the %1 syntax but via {named} tags. I think those are easier to translate, but thats certainly up for debate. I currently just do a string replace on the placeholders before parsing the message. Currently you can pass placeholders via a vararg or via a map.

String msg = "{red}{greeting} stranger. {click:run_command:/warp {spawn}}Click here to get to spawn."
sender.sendRichMessage(msg, "greeting", "Hello", "spawn", "new_spawn");
// kotlin's map idom
send.sendRichMessage(msg, mapOf("greeting" to "Hello", "spawn" to "new_spawn"));

I do agree that using json style curly brackets can be confusing and I think using html like tags like @kashike suggested would be a good idea.

[aikar]

i'm fine with <> too, as [ ] is more common in messages than < >

[MiniDigger]

ok, both the parser (rich message -> bungee components) and the serializer (bungee components -> rich message) works now and uses html like tags

next todo would be patching the chat event (we just need to change the async one, right?).
I have not a real idea yet on how I would want that to look like.

[Andre601]

If I understand this correctly does this implement a method/way to send JSON messages through a more simplified way? Or is this something else? (I base this of the linked issue from the first PR comment)

[MiniDigger]

This PR allows you to send complicated json messages as simple strings, which are way easier to deal with for users and for other plugins.
#1497 (comment) has a good example of what that could look like.
I currently have no time to continue this but I am happy to try to assist with anyone who tries to pick this up.

[MiniDigger]

this is really old, for everyone interested https://github.com/KyoriPowered/adventure-text-minimessage is a thing now