extract_scope documentation details

[brupelo]

Problem description

All docs living here at http://www.sublimetext.com/docs/3/ and also here https://www.sublimetext.com/docs/3/api_reference.html are clear as water to me but even so it's really difficult to figure out how extract_scope works internally and on which cases that API function is relevant when developing new plugins.

Docs says:

extract_scope(point) | Region | Returns the extent of the syntax scope name assigned to the character at the given point.

And after reading it I think, "ermmm... right... ok... :P"

Preferred solution

Ideally you'd add a new section to the http://www.sublimetext.com/docs/3/ package development section explaining this or other type of advanced topics that could clarify the internal behaviour of the public API. I'm a strong believer the name of a public function should be descriptive enough for clients but in tricky cases like this that's just not possible and you need to elaborate about it.

Alternatives

Just providing the relevant information here in this thread allowing the more curious hardcore plugin developers understand how this function works behind the curtains would be also good enough. Not sure whether this function is used very intensively by packages out there.

Additional Information (optional)

I've already asked about this topic few days ago, for me this is really important as I'd like to implement the same functionality than extract_scope using syntect so I could add a similar toggle_command to some scintilla widget that behaved identically to Sublime :)

[brupelo]

@wbond Hi Will, sorry to insist about this one but I'd really like to dispatch it once and for all :)

If you find 3-5 minutes over there eventually, could you please provide a little explanation about the internals of this method? I mean, it doesn't need to be a very detailed explanation dealing with all edge cases... just giving a high-level explanation about the method will be more than good enough and it will allow me to implement it by myself and compare results with the original Sublime's method... once I've got a match 1:1 I could just post my own implementation here or just explaining the algorithm myself :)

Thanks in advance, just for the record I followed your advice of posting these type of question here :)

[brupelo]

@wbond: i've ported syntect to python and i'd like to implement this routine using it. Could you please give me some hints about how sublime does it? Nothing too fancy... A very short explanation will using high level concepts will do. Thanks... Sorry to be so annoying but this is the last element im missing to have toggle comments features on a scintilla widget :’( . Tyvm!!!! 😆

[wbond]

I haven't actually worked on extract_scope before, so I'd have to read over it before I can explain how it works. Right now I'm busy on fixing things from the 3.2 release.

[brupelo]

Doh! 😊... I understand, ty for letting me know. Thing is, usually I'm quite sharp solving puzzles or extracting patterns from blobs of data. But in this case, the fact I'm not a ninja using scopes and selectors won't help neither. Anyway, every little detail or hint you eventually can think of would be really helpful. Guess I'll have to wait more patiently 😇.

Just for the record... It seems limetext equivalent won't behave the same way 🤔

[brupelo]

@wbond sorry to ping again, guess posting here the more relevant source bits of that particular routine is not possible? That way i coul portd the thing to python, check correctness comparing with sublime and eventually creating the relevant documentation myself... So you wouldn't have to waste any time with this low-prio uninteresting ticket... ;-)

[Thom1729]

Maybe I'm missing something, but my impression is that extract_scope:

• Gets the scope at the given point.
• Finds the largest region containing the given point such that the scope at that point is a prefix of the scope of every point in the region.

From an implementation standpoint, it probably starts at the given point and moves leftward until it finds a point that does not match, and likewise rightward. (It probably moves by tokens, not by characters, but the result is the same.)

I seem to recall that there was some weirdness with the "Expand selection to scope" command, which I assume would use extract_scope, but I don't remember the details.

[brupelo]

@Thom1729 Not sure I've understood fully your previous explanation but today I'll try to allocate some time to give it a shot... In case you're also interested to mess around and figure out how this algorithm works, you can use either:

• https://pypi.org/project/pyblime/
  or
• https://pypi.org/project/pysyntect/

Reason why I've insisted so much on knowing the internals of SublimeText extract_scope is because is the last piece of the puzzle I need to be able to use comment.py on pyblime... which I consider one of the most important features when coding stuff... In fact, if you can't comment/uncomment code in a text editor the whole thing becomes pretty useless to me :P

I'm aware I've been quite pressing about this one so I wouldn't blame Will thinking "here's this annoying guy again", I wouldn't blame him, sorry about that :D

Anyway Thom, thanks for the explanation, let's see.

[brupelo]

@Thom1729 Here's a little broken attempt:

import re
import textwrap
from itertools import groupby
import operator
import sublime
import sublime_plugin
from sublime import Region


class MyExtractScope(sublime_plugin.TextCommand):

    def extract_scope(self, pt):
        view = self.view
        size = view.size()
        scope = view.scope_name(pt)

        def expand(pos, offset):
            while view.match_selector(pos, scope):
                pos = pos + offset
                if pos < 0 or pos > size:
                    break

            return pos + 1 if offset < 0 else pos - 1

        return Region(expand(pt, -1), expand(pt, +1))

    def test(self, pt):
        f_mine = self.extract_scope
        f_subl = self.view.extract_scope
        r1 = f_mine(pt)
        r2 = f_subl(pt)
        if r1 != r2:
            raise Exception("position={} => {} vs {}".format(pt, r1, r2))

    def run(self, edit, block=False):
        errors = 0
        size = self.view.size()
        for i in range(size):
            try:
                self.test(i)
            except Exception as e:
                print(e)
                errors+=1

        print("Total:", size, "Ok:", size-errors, "Errors:", errors)

If you bind that command to some key and run it on the view where you pasted the code itself you'll see a lot of errors were spawned. Said otherwise, this isn't how extract_scope works :)

[BenjaminSchaaf]

@brupelo It's not expanding to adjacent scopes by using the scope as a selector, but simply checking if the adjacent scopes begin identically.

[BenjaminSchaaf]

Fixed in build 4135.