Port to Python 3 #844

Aniket21mathur · 2019-07-03T06:43:45Z

Replaces #805

Aniket21mathur · 2019-07-03T08:43:31Z

I am not able to test the changes.
What I did-

Installed sugar-artwork from head.
Installed sugar-datastore from python3 branch.
Installed toolkit for python3.
Installed sugar @ b9d773f.
Insured that the modules are in dist-packages.
Used https://github.com/aniket21mathur/telepathy3 for telepathy.

When I start sugar, it fails. When I check .sugar for logs, the logs directory is not build, hence not able to proceed further.

Aniket21mathur · 2019-07-03T09:07:45Z

I am getting this as a traceback on starting activities from Ubuntu terminal using sugar-activity3 .

dbus.exceptions.DBusException: org.freedesktop.DBus.Python.TypeError: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/dbus/service.py", line 707, in _message_cb
    retval = candidate_method(self, *args, **keywords)
  File "/usr/local/lib/python3.6/dist-packages/carquinyol/datastore.py", line 308, in create
    self._metadata_store.store(uid, props)
  File "/usr/local/lib/python3.6/dist-packages/carquinyol/metadatastore.py", line 24, in store
    self._set_property(uid, key, value, md_path=metadata_path)
  File "/usr/local/lib/python3.6/dist-packages/carquinyol/metadatastore.py", line 51, in _set_property
    f.write(value)
TypeError: write() argument must be str, not dbus.Int32

1562144740.799171 WARNING root: DSObject was deleted without cleaning up first. Call DSObject.destroy() before disposing it.

quozl · 2019-07-08T07:34:36Z

That traceback is in sugar-datastore module, and from the paths it is ported to Python 3. You should have found it in datastore.log, is that right? You had best validate the arguments sent in the API call. Try dbus-monitor and look for the message that triggered it.

Aniket21mathur · 2019-07-09T03:29:03Z

You should have found it in datastore.log, is that right?

No, I got it when I tried to run the Chat activity from Ubuntu terminal, using sugar-activity3 . in the root directory. Surprising, but happens occasionally, I don't know what causes it, I am still working on it.

Thanks!

quozl · 2019-07-09T05:02:45Z

Thanks. That's not a common usage. No child is expected to do that. Only if activities fail to run in Sugar Terminal or from Sugar home view, list view, or sharing, would it be considered blocking. So I'll move back to your previous comment; Sugar does not start, and there are no logs. The cause must be uncovered. Any progress?

Aniket21mathur · 2019-07-09T06:19:51Z

Thanks. That's not a common usage. No child is expected to do that. Only if activities fail to run in Sugar Terminal or from Sugar home view, list view, or sharing, would it be considered blocking

Thanks, noted.

Any progress?

I have decided to first work on sugar-datastore Python 3 port, as sugar depends on it. We will not able to test the changes in sugar fairly until and unless we have all the dependencies working fine. What do you suggest?

quozl · 2019-07-09T08:48:10Z

I have decided to first work on sugar-datastore Python 3 port, as sugar depends on it. We will not able to test the changes in sugar fairly until and unless we have all the dependencies working fine. What do you suggest?

I don't think I need to prioritise Sugar vs Datastore. It's up to you. They are considerably different; the former is a GTK application, the latter is a D-Bus service. We have no other D-Bus services in our sources.

However, Datastore could be run in Python 3 and tested against either a Python 2 or Python 3 Sugar, and vice versa. I don't think there should be any problem mixing Python versions across the D-Bus barrier.

Aniket21mathur · 2019-07-10T16:12:46Z

I am still not able to test these changes, Sugar does not start, .sugar builds but only with debug file, logs is not built. I am not able to proceed further, as I don't get anything in the form of logs.

Thanks!

quozl · 2019-07-11T04:44:16Z

When you say Sugar does not start, what have you observed to conclude that?

How are you starting Sugar?

I suggest using strace. Read about the command in man strace. Then try something like this;

temporarily move /usr/bin/sugar to /usr/bin/sugar.real,
add a shell script /usr/bin/sugar containing;

#!/bin/sh
strace -o /tmp/sugar.strace -f /usr/bin/sugar.real > /tmp/sugar.out 2> /tmp/sugar.err

start Sugar once more; the results will be in those three files in /tmp

Aniket21mathur · 2019-07-11T04:52:55Z

How are you starting Sugar?

I am trying to log in into the Sugar Desktop from Ubuntu login, when I press enter after typing the password, the cursor appears with a black screen, and nothing happens further.

I suggest using strace

Thanks, I will try that :-)

quozl · 2019-07-11T05:56:41Z

Re: d501d7f. Heh. 😁 Now, is there something we can do to make it clearer that this has gone wrong?

Aniket21mathur · 2019-07-11T06:11:04Z

temporarily move /usr/bin/sugar to /usr/bin/sugar.real,

Your suggestion made me realise that actually sugar is being called from /usr/bin instead of /usr/local/bin, the other one is the Python 3 one.

I have removed dependencies for Python 2 version for testing, that was the reason for the failure of sugar.

Now I am able to start sugar.

Observations:-

Journal is not working, nothing gets displayed on the jounal, nor does the your journal is empty message.
Not able to switch to neigbourhood and group view, pressing fnc + f1 or fnc + f2 directs to the home view.
Activity startup works fine.
home view, frame view works fine.
Nothing in the logs as at d501d7f.

Re: d501d7f. Heh. Now, is there something we can do to make it clearer that this has gone wrong?

I got a traceback in the logs, regarding that 😅

@quozl this was the traceback

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/jarabe/journal/volumestoolbar.py", line 205, in _set_up_volumes
    self._add_button(mount)
  File "/usr/local/lib/python3.6/dist-packages/jarabe/journal/volumestoolbar.py", line 235, in _add_button
    button = VolumeButton(mount)
  File "/usr/local/lib/python3.6/dist-packages/jarabe/journal/volumestoolbar.py", line 324, in __init__
    self.props.xo_color = get_mount_color(self._mount)
  File "/usr/local/lib/python3.6/dist-packages/jarabe/journal/misc.py", line 424, in get_mount_color
    sha_hash.update(mount_name)
TypeError: Unicode-objects must be encoded before hashing

quozl · 2019-07-11T06:22:16Z

Thanks. Should you ever need to, rather than remove Sugar for Python 2, whatever calls /usr/bin/sugar can be changed to call /usr/local/bin/sugar. It is the /usr/share/xsessions/sugar.desktop file added to the system by the sugar-session binary package. This originally came from data/sugar.desktop in the Sugar repository, where it has no path, but the path was added by the Debian developers in the debian/sugar.desktop file in the sugar source package. This makes it harder to use both a packaged Sugar and a /usr/local Sugar at the same time.

But it's still odd that you got a black screen and nothing in logs for trying to start a Python 2 build of Sugar on the same system that contained a /usr/local Python 3 build of Sugar. I think there's something more to that than meets the eye.

Do keep diagnosing why Journal, Neighbourhood and Group views are not working.

Aniket21mathur · 2019-07-11T06:38:37Z

Thanks.

Do keep diagnosing why Journal, Neighbourhood and Group views are not working.

Yes, I am working on it, @pro-panda your suggestions will be valuable here, as you instantiated these changes 😁

quozl · 2019-07-11T06:44:10Z

It could easily be something in the toolkit that isn't used by activities. I suggest instrumenting the Sugar code with logging and use an attached debugger to determine state of relevant objects.

Aniket21mathur · 2019-07-11T06:48:02Z

It could easily be something in the toolkit that isn't used by activities. I suggest instrumenting the Sugar code with logging and use an attached debugger to determine state of relevant objects.

Thanks, currently I am looking at /sugar/src/jarabe/view/keyhandler.py .

ghost · 2019-07-11T14:59:33Z

On Wed, Jul 10, 2019 at 09:44:18PM -0700, James Cameron wrote: I suggest using strace. Read about the command in man strace. Then try something like this; • temporarily move /usr/bin/sugar to /usr/bin/sugar.real, • add a shell script /usr/bin/sugar containing; #!/bin/sh strace -o /tmp/sugar.strace -f /usr/bin/sugar.real > /tmp/sugar.out 2> /tmp/sugar.err • start Sugar once more; the results will be in those three files in /tmp

Optionally, and especially if you find yourself doing anything interactive at the shell, you can create a log of everything displayed in the terminal by wrapping your session with an instance of the script(1) command: http://manpages.ubuntu.com/manpages/disco/en/man1/script.1.html Directing specific output to specific files as above is the cleanest way to do it. You know which output came from STDOU (>), which from STDERR (2>), and which from strace (-o). It does require more files, though, which can compound the difficulty of tracking what output corresponds with what iteration. Committing such ephemeral output files to version control isn't really appropriate, but one needs some way of distinguishing, for instance, output from before a change was made versus output after a change was made. I often will indicate the before/after origins of the output by changing the filenames. When there are multiple filenames like this, I all too often neglect to change one of them to match the others, instead overwriting a file from a previous run. So, the output is more muddled, but I find it more tractable to have just one file per run, recorded with the 'script' command. One thing to be very careful of, though: If you interactively start recording with script, be sure to 'exit' out of the subshell to close out the logfile, or it will grow huge! Another approach to normalizing your filenames across trials for ephemeral output like this is to datestamp the filenames. For instance, here I append the number of seconds since the epoch to each filename: strace -o /tmp/sugar.strace-$(date +%s) -f /usr/bin/sugar.real > /tmp/sugar.out-$(date +%s) 2> /tmp/sugar.err-$(date +%s) Finally, another approach to keep the output from multiple runs separate is to put the results from each run in its own directory. This has a similar drawback as using multiple files per run, though, in that you have to be particularly mindful of your working directory all the time lest you inadvertently overwrite results from a previous iteration. Another way to look at this might be: If you have reproducible behavior that you would like to document and communicate to others, use redirects. For internal, interactive use, where you're not yet sure what's happening, where, or why, it's better to have *some* log than a perfect log, and so you might find 'script' handy. Hope that helps,

…

-- D. Joe

quozl · 2019-07-11T21:47:12Z

Thanks Joe!

Another way to get logs from early failure of /usr/bin/sugar is to change the sugar.desktop file to start /usr/bin/xterm or another terminal instead, and then when you log in you'll get an terminal instance, and you can then type sugar.

Aniket21mathur · 2019-07-14T04:58:21Z

Testing configuration~

sugar, at bcc2c3b
sugar-toolkit at sugarlabs/sugar-toolkit-gtk3@7601a93 (Python 3 version)
sugar-datastore (HEAD)
sugar-artwork (HEAD)
Chat Activity (Both Python 2 and Python 3 )

Observations~

Journal works fine.
Home view works fine.
Activity startup works fine.
Neighbourhood view works fine.
Group View, make friend works fine.
Collaboration works fine.
File transfer works fine.

Issue~

Timestamp in the journal is coming in years.

Note~
Changes at sugarlabs/sugar-toolkit-gtk3#419 need to be incorporated for proper working.

Thanks!

quozl · 2019-07-16T00:00:40Z

Thanks, good progress.

Aniket21mathur · 2019-07-16T09:31:09Z

Timestamp in the journal is coming in years.

@quozl @pro-panda I am not able to figure out how to fix it, as a work around I looked into the files of sugar and datastore having the keyword timestamp but not able to figure out what's wrong. Need help.

quozl · 2019-07-17T00:23:24Z

Timestamp in the journal is coming in years.

Please describe the symptom? Of particular interest will be (a) the user operations that lead up to the event, (b) the value shown in the second last column of the Journal, and (c) the value shown in the "Date" field of the detailed view of the journal object.

For instance; (a) open Write activity, stop, open Journal, show detailed view, (b) "Seconds ago", and (c) "Date: 17/07/19".

Aniket21mathur · 2019-07-17T03:39:37Z

(a) the user operations that lead up to the event,

Open and close any activity, creating an instance in journal.

(b) the value shown in the second last column of the Journal

"0 years ago"

he value shown in the "Date" field of the detailed view of the journal object.

By this did you mean detail view of an activity instance in journal? Sorry it's not clear to me.

Thanks.

quozl · 2019-07-17T04:23:13Z

It's a fault in the toolkit. Watch me test by hand;

% python3
Python 3.7.3 (default, Apr  3 2019, 05:39:12) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from sugar3.util import timestamp_to_elapsed_string
>>> import time
>>> x = time.time() - 300  # five minutes ago
>>> timestamp_to_elapsed_string(x)
'0 years ago'
>>>

vs

% python2
Python 2.7.16 (default, Apr  6 2019, 01:42:57) 
[GCC 8.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from sugar3.util import timestamp_to_elapsed_string
>>> import time
>>> x = time.time() - 300  # five minutes ago
>>> timestamp_to_elapsed_string(x)
'5 minutes ago'

quozl · 2019-07-17T04:24:06Z

By this did you mean detail view of an activity instance in journal?

Yes. You'll probably find no issue with the date shown there.

Aniket21mathur · 2019-07-17T07:14:55Z

Thanks. Made a fix in sugarlabs/sugar-toolkit-gtk3#420, please test.

quozl · 2019-07-17T07:32:45Z

Tested and merged.

Has this pull request now reached full functionality?

Aniket21mathur · 2019-07-17T07:36:20Z

Tested and merged.

Thanks.

Has this pull request now reached full functionality?

Yes. I tested the changes from my side, It will be really good if you test again. 😁

Aniket21mathur · 2019-07-18T08:06:54Z

@pro-panda it will be great if you also review and test it. :-)

quozl · 2019-07-19T07:39:59Z

Can't test, because can't resolve dependencies, because there's no package python3-gwebsockets yet.

Aniket21mathur · 2019-07-19T08:14:58Z

an't test, because can't resolve dependencies, because there's no package python3-gwebsockets yet.

Yes agreed, I am still working on it. Thanks.

Aniket21mathur · 2019-08-02T06:31:37Z

@pro-panda @quozl May you please test this pr now?
Thanks!

quozl · 2019-08-02T06:41:17Z

Given the complexity of native install, and the difficulty others have had in reproducing the work so far, we really should try for packages. How is progress on toolkit packages for Python 3?

* remove call to sys.setdefaultencoding default encoding is 'utf-8' by default in Python 3 * remove use of statvfs statvfs does not exist in python 3 * TODO was fixed in af6ce2e * Remove unused function * flake fixes * Fix unicode encoding error * Use a list to iterate over, supporting Python 3 * Encode 'str' instance into 'utf-8' Co-authored-by: Aniket21mathur <[email protected]>

quozl · 2019-08-13T06:29:34Z

Although testing is incomplete, there's other pull requests coming into Sugar that need to be based off this work.

Aniket21mathur mentioned this pull request Jul 3, 2019

Port to Python 3 #805

Closed

This comment has been minimized.

Sign in to view

quozl mentioned this pull request Jul 30, 2019

Release v0.7 sugarlabs/gwebsockets#2

Closed

5 tasks

quozl merged commit aa18879 into sugarlabs:master Aug 13, 2019

Port to Python 3 #844

Port to Python 3 #844

Conversation

Aniket21mathur commented Jul 3, 2019 • edited by quozl Loading

Aniket21mathur commented Jul 3, 2019 • edited Loading

Aniket21mathur commented Jul 3, 2019

quozl commented Jul 8, 2019

Aniket21mathur commented Jul 9, 2019

quozl commented Jul 9, 2019

Aniket21mathur commented Jul 9, 2019 • edited Loading

quozl commented Jul 9, 2019

Aniket21mathur commented Jul 10, 2019

quozl commented Jul 11, 2019

Aniket21mathur commented Jul 11, 2019 • edited Loading

quozl commented Jul 11, 2019 • edited Loading

Aniket21mathur commented Jul 11, 2019 • edited Loading

quozl commented Jul 11, 2019

Aniket21mathur commented Jul 11, 2019

quozl commented Jul 11, 2019 • edited Loading

Aniket21mathur commented Jul 11, 2019

ghost commented Jul 11, 2019 via email

quozl commented Jul 11, 2019

This comment has been minimized.

This comment has been minimized.

Aniket21mathur commented Jul 14, 2019 • edited Loading

quozl commented Jul 16, 2019

Aniket21mathur commented Jul 16, 2019 • edited Loading

quozl commented Jul 17, 2019

Aniket21mathur commented Jul 17, 2019 • edited Loading

quozl commented Jul 17, 2019

quozl commented Jul 17, 2019

Aniket21mathur commented Jul 17, 2019

quozl commented Jul 17, 2019

Aniket21mathur commented Jul 17, 2019

Aniket21mathur commented Jul 18, 2019

quozl commented Jul 19, 2019

Aniket21mathur commented Jul 19, 2019

Aniket21mathur commented Aug 2, 2019

quozl commented Aug 2, 2019

quozl commented Aug 13, 2019

Aniket21mathur commented Jul 3, 2019 •

edited by quozl

Loading

Aniket21mathur commented Jul 3, 2019 •

edited

Loading

Aniket21mathur commented Jul 9, 2019 •

edited

Loading

Aniket21mathur commented Jul 11, 2019 •

edited

Loading

quozl commented Jul 11, 2019 •

edited

Loading

Aniket21mathur commented Jul 11, 2019 •

edited

Loading

quozl commented Jul 11, 2019 •

edited

Loading

Aniket21mathur commented Jul 14, 2019 •

edited

Loading

Aniket21mathur commented Jul 16, 2019 •

edited

Loading

Aniket21mathur commented Jul 17, 2019 •

edited

Loading