Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add generate lemmy instance list gradle task #884

Merged
merged 7 commits into from
Jun 26, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
* [Kotlin](#kotlin)
* [Code quality](#code-quality)
* [Adding translations](#adding-translations)
* [Updating the instance list](#updating-the-instance-list)
* [Generate compose compiler metrics](#generate-compose-compiler-metrics)
<!-- TOC -->

<!-- prettier-ignore-end -->
Expand Down Expand Up @@ -73,3 +75,25 @@ prettier --write "*.md" "*.yml"`
You can find the translations in the `app/src/main/res/values-{locale}/strings.xml` file.
You can open it in android studio, right click and click open translations editor or you can
directly edit the files.

## Updating the instance list

There is a custom gradle task that generates all the lemmy instances that this app directly supports.
It updates the lemmy instances list in DefaultInstances.kt and the AndroidManifest.
It uses the fediverse api and filters on the monthly users.
You can run it by doing

```shell
./gradlew app:updateInstances -no-configuration-cache
MV-GH marked this conversation as resolved.
Show resolved Hide resolved
```

## Generate compose compiler metrics

You can generate the compose compiler metrics by executing the following gradle task.

```shell
./gradlew assembleRelease --rerun-tasks -P com.jerboa.enableComposeCompilerReports=true
```

Then you will find the metrics in `app/build/compose_metrics` directory.
See [this link for more information on these metrics](https://github.com/androidx/androidx/blob/androidx-main/compose/compiler/design/compiler-metrics.md)
2 changes: 2 additions & 0 deletions app/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ plugins {
id("androidx.baselineprofile")
}

apply(from = "update_instances.gradle.kts")

android {
compileSdk = 33

Expand Down
71 changes: 57 additions & 14 deletions app/src/main/AndroidManifest.xml
Original file line number Diff line number Diff line change
Expand Up @@ -59,26 +59,69 @@
<data android:mimeType="image/*" />
</intent-filter>

<!-- The main instances-->
<intent-filter>
<action android:name="android.intent.action.VIEW" />
<category android:name="android.intent.category.DEFAULT" />
<category android:name="android.intent.category.BROWSABLE" />
<data android:scheme="https"/>
<!--#AUTO_GEN_INSTANCE_LIST_DO_NOT_TOUCH#-->
<data android:host="lemmy.world"/>
<data android:host="lemmy.ml"/>
<data android:host="beehaw.org"/>
<data android:host="feddit.de" />
<data android:host="feddit.it" />
<data android:host="lemmy.ca" />
<data android:host="lemmy.ml" />
<data android:host="lemmy.one" />
<data android:host="lemmy.world" />
<data android:host="lemmygrad.ml" />
<data android:host="midwest.social" />
<data android:host="mujico.org" />
<data android:host="sh.itjust.works" />
<data android:host="slrpnk.net" />
<data android:host="sopuli.xyz" />
<data android:host="szmer.info" />
<data android:host="feddit.de"/>
<data android:host="sh.itjust.works"/>
<data android:host="www.hexbear.net"/>
<data android:host="lemmynsfw.com"/>
<data android:host="lemmy.ca"/>
<data android:host="lemmy.one"/>
<data android:host="lemm.ee"/>
<data android:host="lemmy.fmhy.ml"/>
<data android:host="lemmy.dbzer0.com"/>
<data android:host="lemmy.blahaj.zone"/>
<data android:host="lemmygrad.ml"/>
<data android:host="programming.dev"/>
<data android:host="discuss.tchncs.de"/>
<data android:host="sopuli.xyz"/>
<data android:host="lemmy.sdf.org"/>
<data android:host="vlemmy.net"/>
<data android:host="midwest.social"/>
<data android:host="aussie.zone"/>
<data android:host="startrek.website"/>
<data android:host="infosec.pub"/>
<data android:host="feddit.uk"/>
<data android:host="dormi.zone"/>
<data android:host="feddit.it"/>
<data android:host="pawb.social"/>
<data android:host="feddit.nl"/>
<data android:host="burggit.moe"/>
<data android:host="slrpnk.net"/>
<data android:host="lemmy.nz"/>
<data android:host="feddit.dk"/>
<data android:host="delraymisfitsboard.com"/>
<data android:host="mander.xyz"/>
<data android:host="reddthat.com"/>
<data android:host="feddit.cl"/>
<data android:host="lemmy.zip"/>
<data android:host="lemmy.pt"/>
<data android:host="dataterm.digital"/>
<data android:host="szmer.info"/>
<data android:host="latte.isnot.coffee"/>
<data android:host="lemmy.eco.br"/>
<data android:host="monyet.cc"/>
<data android:host="exploding-heads.com"/>
<data android:host="waveform.social"/>
<data android:host="lemmy.tedomum.net"/>
<data android:host="enterprise.lemmy.ml"/>
<data android:host="pathofexile-discuss.com"/>
<data android:host="iusearchlinux.fyi"/>
<data android:host="yiffit.net"/>
<data android:host="ttrpg.network"/>
<data android:host="lemmyrs.org"/>
<data android:host="sub.wetshaving.social"/>
<data android:host="monero.town"/>
<data android:host="bakchodi.org"/>
<data android:host="geddit.social"/>
<!--#INSTANCE_LIST_END#-->
</intent-filter>
</activity>
</application>
Expand Down
60 changes: 60 additions & 0 deletions app/src/main/java/com/jerboa/DefaultInstances.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
package com.jerboa

val DEFAULT_LEMMY_INSTANCES = arrayOf(
"lemmy.world", // 13856 monthly users
"lemmy.ml", // 4206 monthly users
"beehaw.org", // 3730 monthly users
"feddit.de", // 2300 monthly users
"sh.itjust.works", // 2215 monthly users
"www.hexbear.net", // 1603 monthly users
"lemmynsfw.com", // 1122 monthly users
MV-GH marked this conversation as resolved.
Show resolved Hide resolved
"lemmy.ca", // 1092 monthly users
"lemmy.one", // 987 monthly users
"lemm.ee", // 984 monthly users
"lemmy.fmhy.ml", // 944 monthly users
"lemmy.dbzer0.com", // 782 monthly users
"lemmy.blahaj.zone", // 769 monthly users
"lemmygrad.ml", // 629 monthly users
"programming.dev", // 617 monthly users
"discuss.tchncs.de", // 564 monthly users
"sopuli.xyz", // 561 monthly users
"lemmy.sdf.org", // 476 monthly users
"vlemmy.net", // 398 monthly users
"midwest.social", // 396 monthly users
"aussie.zone", // 373 monthly users
"startrek.website", // 317 monthly users
"infosec.pub", // 286 monthly users
"feddit.uk", // 286 monthly users
"dormi.zone", // 269 monthly users
"feddit.it", // 260 monthly users
"pawb.social", // 242 monthly users
"feddit.nl", // 197 monthly users
"burggit.moe", // 185 monthly users
"slrpnk.net", // 178 monthly users
"lemmy.nz", // 166 monthly users
"feddit.dk", // 159 monthly users
"delraymisfitsboard.com", // 158 monthly users
"mander.xyz", // 152 monthly users
"reddthat.com", // 152 monthly users
"feddit.cl", // 123 monthly users
"lemmy.zip", // 118 monthly users
"lemmy.pt", // 100 monthly users
"dataterm.digital", // 86 monthly users
"szmer.info", // 85 monthly users
"latte.isnot.coffee", // 85 monthly users
"lemmy.eco.br", // 85 monthly users
"monyet.cc", // 85 monthly users
"exploding-heads.com", // 83 monthly users
"waveform.social", // 78 monthly users
"lemmy.tedomum.net", // 76 monthly users
"enterprise.lemmy.ml", // 72 monthly users
"pathofexile-discuss.com", // 71 monthly users
"iusearchlinux.fyi", // 65 monthly users
"yiffit.net", // 62 monthly users
"ttrpg.network", // 61 monthly users
"lemmyrs.org", // 59 monthly users
"sub.wetshaving.social", // 57 monthly users
"monero.town", // 54 monthly users
"bakchodi.org", // 53 monthly users
"geddit.social", // 50 monthly users
)
23 changes: 3 additions & 20 deletions app/src/main/java/com/jerboa/Utils.kt
Original file line number Diff line number Diff line change
Expand Up @@ -83,23 +83,6 @@ val gson = Gson()
const val DEBOUNCE_DELAY = 1000L
const val MAX_POST_TITLE_LENGTH = 200

val DEFAULT_LEMMY_INSTANCES = listOf(
"beehaw.org",
"feddit.de",
"feddit.it",
"lemmy.ca",
"lemmy.ml",
"lemmy.one",
"lemmy.world",
"lemmygrad.ml",
"midwest.social",
"mujico.org",
"sh.itjust.works",
"slrpnk.net",
"sopuli.xyz",
"szmer.info",
)

// convert a data class to a map
fun <T> T.serializeToMap(): Map<String, String> {
return convert()
Expand Down Expand Up @@ -1320,9 +1303,9 @@ fun getLangPreferenceDropdownEntries(ctx: Context): Map<Locale, String> {
val localeList = getLocaleListFromXml(ctx)
val map = mutableMapOf<Locale, String>()

for (a in 0 until localeList.size()) {
localeList[a].let {
it?.let { it1 -> map.put(it, it.getDisplayName(it)) }
for (i in 0 until localeList.size()) {
localeList[i]?.let {
map.put(it, it.getDisplayName(it))
}
}
return map
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -293,9 +293,9 @@ fun InboxTabs(
items(
replies,
key = { reply -> reply.comment_reply.id },
) { crv ->
) { commentReplyView ->
CommentReplyNode(
commentReplyView = crv,
commentReplyView = commentReplyView,
onUpvoteClick = { cr ->
account?.also { acct ->
inboxViewModel.likeReply(
Expand Down
142 changes: 142 additions & 0 deletions app/update_instances.gradle.kts
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
import java.io.OutputStreamWriter
import java.net.URL
import java.net.HttpURLConnection

// We can't import libraries here for some reason, so we must use what is provided
// by gradle, which isn't much. The groovy JSON library is meant for use by groovy code,
// so we need some creativity to use it in Kotlin.
import org.apache.groovy.json.internal.LazyMap
import groovy.json.JsonOutput
import groovy.json.JsonSlurper

// All lemmy instances with at least this amount of monthly active users will be included.
val minimumMAU = 50


val endpointUrl = "https://api.fediverse.observer/"
val instancesFilePath = "src/main/java/com/jerboa/DefaultInstances.kt"
val manifestPath = "src/main/AndroidManifest.xml"
val START_TAG = "<!--#AUTO_GEN_INSTANCE_LIST_DO_NOT_TOUCH#-->"
val END_TAG = "<!--#INSTANCE_LIST_END#-->"
val IDENT = 14


// Some extension methods to make the JsonSlurper output easier to process
fun LazyMap.getMap(key: String): LazyMap {
return this[key] as LazyMap
}

fun LazyMap.getArray(key: String): ArrayList<*> {
return this[key] as ArrayList<*>
}

@Suppress("UNCHECKED_CAST")
fun <T> LazyMap.getAs(key: String): T {
return this[key] as T
}

// Run this as `./gradlew app:updateInstances -no-configuration-cache`
tasks.register("updateInstances") {
description = "Fetches a list of popular Lemmy instances and writes it to the DefaultInstances.kt file"

doFirst {
// Get sorted list of nodes
val nodes = getData()
.getMap("data")
.getArray("nodes")
.map {
val name = (it as LazyMap)["domain"] as String
val users = it["active_users_monthly"] as Int?

Pair(name, users ?: 0)
}
.filter {
it.second >= minimumMAU
}
.sortedBy {
it.second
}
.reversed()

updateInstanceList(nodes)
updateManifest(nodes.map { it.first })
}
}


fun getData(): LazyMap {
val url = URL(endpointUrl)
val query = """
{
nodes(softwarename: "lemmy") {
domain
active_users_monthly
}
}"""

// Format JSON request body
val body = JsonOutput.toJson(mapOf("query" to query))

// Create POST request
val req = url.openConnection() as HttpURLConnection
req.requestMethod = "POST"
req.doOutput = true
req.setRequestProperty("Content-Type", "application/json")

// Write body to request
OutputStreamWriter(req.outputStream, "UTF-8").use {
it.write(body)
}

// Get response and JSON parse it
return JsonSlurper().parse(req.inputStream.reader()) as LazyMap
}

fun updateInstanceList(nodes: List<Pair<String, Int>>) {
// Create output file and write header
val outFile = file(instancesFilePath)
outFile.writeText(
"""package com.jerboa

val DEFAULT_LEMMY_INSTANCES = arrayOf(
"""
)

// Write each node's name, one per line
for (n in nodes) {
outFile.appendText(" \"${n.first}\", // ${n.second} monthly users\n")
}

outFile.appendText(")\n")
}


fun updateManifest(list: List<String>) {
val manifest = file(manifestPath)
val lines = manifest.readLines()
manifest.writeText("")

var skip = false

for (line in lines) {
if (line.trim() == START_TAG) {
skip = true
manifest.appendText(" ".repeat(IDENT) + START_TAG)
manifest.appendText(genManifestHosts(list))
manifest.appendText(" ".repeat(IDENT) + END_TAG + System.lineSeparator())
} else if (line.trim() == END_TAG) {
skip = false
} else if (!skip) {
manifest.appendText(line + System.lineSeparator())
}
}
}

fun genManifestHosts(list: List<String>): String {
return list.joinToString(
separator = System.lineSeparator(),
prefix = System.lineSeparator(),
postfix = System.lineSeparator(),
) { " ".repeat(IDENT) + "<data android:host=\"$it\"/>" }
}

4 changes: 0 additions & 4 deletions build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,3 @@ subprojects {
}
}
}

tasks.register("clean", Delete::class) {
delete(rootProject.buildDir)
}