Using cache in filtering script?

[HristoRaykov]

I have a filter script that maps numeric ids to names and it works. The problem is that the dictionary with ids is about 8000 entries and it initiates every time the filter is executed. Is there a way to cache the dictionary and access it inside the script?

[martinrotter]

can you show off your script for me? it would likely help to get me an idea of how could such a caching work

[HristoRaykov]

I'm using EDGAR Latest filings RSS feed https://www.sec.gov/cgi-bin/browse-edgar?action=getcurrent. I am mapping 10 digits company CIK numbers from the title (the one in brackets) to corresponding trading symbols for ones that have such from here https://www.sec.gov/file/company-tickers-exchange.

var ciks = {
    "0000320193": {"name": "Apple Inc.", "symbol": "AAPL", "exchange": "Nasdaq"},
    "0001045810": {"name": "NVIDIA CORP", "symbol": "NVDA", "exchange": "Nasdaq"},
    "0000789019": {"name": "MICROSOFT CORP", "symbol": "MSFT", "exchange": "Nasdaq"},
    "0001018724": {"name": "AMAZON COM INC", "symbol": "AMZN", "exchange": "Nasdaq"},
    "0001652044": {"name": "Alphabet Inc.", "symbol": "GOOGL", "exchange": "Nasdaq"}
//     much more entries here around 8000
}
var reg = /\d{10}/;

function filterMessage() {
    var match = reg.exec(msg.title);
    if (match) {
        var cik = match[0];
        if (cik in ciks) {
            var data = ciks[cik];
            var symbol = data["symbol"];
            var exchange = data["exchange"];
            msg.title = `${symbol}: ${exchange} - ${msg.title}`;
        }
        else {
            msg.title = `Not Listed - ${msg.title}`;
        }
    }

    return MessageObject.Accept;
}

ciks dictionary initiates every time the script is run and it is slow. Can I cache it so to initiate it only ones and share it b/w different calls of the script?

[martinrotter]

can you provide full script for me to test?

[HristoRaykov]

RSS url: https://www.sec.gov/cgi-bin/browse-edgar?action=getcurrent&CIK=&type=&company=&dateb=&owner=include&start=0&count=40&output=atom

The code is too long so attaching it as a file.
edgar_filter.txt

[martinrotter]

OK, so I checked how exactly I have this implemented.

You kind of discovered a "bug". You see now it is implemented via two nested loops:

• outer loop runs over the messages (from a feed)
• inner loop applies the filters for the message one by one

So if you have lets say two messages (A, B) and two filters (X, Y), then execution is like this:
A->X, A->Y, B->X, B->Y

So filters are evaluated in javascript in "interleaving" fashion, so any "caching" is rather problematic. I would have to either switch the loops or make sure to create separate JS environment for each script (which would be a problem for people who have huge amount of filters).

I will have to think about it for some time.

[HristoRaykov]

I understand. Thank you for the quick reply!