get jobs from IBM Q backend #501

ewinston · 2018-05-24T10:22:06Z

This PR allows one to get IBMQJob objects for previously submitted jobs to an IBM Q backend.

job_list = backend.jobs()
job = backend.retrieve_job(job_id)

Description

TODO:

pass tests
skip keyword isn't implemented

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have read the CONTRIBUTING document.
I have added tests to cover my changes.
All new and existing tests passed.

ajavadia · 2018-06-10T20:31:56Z

@ewinston I think this requires an update to IBMQuantumExperience 1.9.2 (should go in requirements.txt)

ajavadia · 2018-06-10T20:41:59Z

qiskit/_result.py

-            self._result['result'] += other._result['result']
-            return self
-        else:
-            raise QISKitError('Result objects have different configs and cannot be combined.')


Don't we still need to do some checks in order to make sure the results can be combined, and if not raise this QISKitError? For example one of the checks is to make sure they come from the same backend. Previously this was done based on qobj, but I think the result has enough information for this check too.

Previously it checked that the 'config' section of the qobj matched, however, perhaps we can just match the backend_name since I don't think 'shots' or 'max_credits' mismatch should raise an error.

ajavadia · 2018-06-10T21:22:36Z

qiskit/_result.py

-        else:
-            raise QISKitError('Result objects have different configs and cannot be combined.')
+        self._result['result'] += other._result['result']
+        return self

    def __add__(self, other):


Does the docstring of this method need updating? I think result doesn't have a qobj_id anymore.

ajavadia · 2018-06-10T21:32:29Z

test/python/test_reordering.py

-from qiskit.wrapper import register, available_backends, get_backend, execute
+from qiskit import QuantumJob
+from qiskit.wrapper import register, available_backends, get_backend
+from qiskit.wrapper import compile as qcompile


can we keep this compile instead of qcompile for consistency with the rest of the code base? it won't collide with the compile keyword after this import. it has been used as compile(), for example in wrapper.py.

I thought this raised a style or linting flag but I'll retry.

ajavadia · 2018-06-10T21:40:01Z

qiskit/backends/ibmq/ibmqbackend.py

+        Args:
+            limit (int): number of jobs to retrieve
+            skip (int): starting index of retrieval
+            status (None or JobStatus): only get jobs with this status.


would it make sense to make this string for easier use? e.g. "running", "done", "queued", ..

if not, then I think this docstring should be updated with some examples, e.g. JobStatus.RUNNING, JobStatus.Done, JobStatus.QUEUED

Since this questions the use of enumerated types perhaps the docstring can just be updated for now.

Actually, it looks easy to also allow the string representation.

ajavadia · 2018-06-10T21:52:56Z

qiskit/_result.py

@@ -33,7 +33,6 @@ class Result(object):

    Internal::

-        qobj =  { -- the quantum object that was complied --}
        result = {


I think now that Result._qobj is removed, we can unpack Result._result, and not have so many levels of indirection. That means having Result._job_id, Result._status, Result._result, where the latter contains all the data.

This sounds ok to me.

Maybe we can leave this for a subsequent PR since it's not directly related to this one?

yeah i think we should leave this, also because it's not clear how qobj and result may be represented together later.
I think the most important thing in this PR is to not give errors when metadata is unavailable from a past job.

ajavadia · 2018-06-10T22:14:55Z

qiskit/backends/ibmq/ibmqjob.py

+        if 'metadata' in circuit_result:
+            circ = circuit_result['metadata'].get('compiled_circuit')
+        else:
+            raise QISKitError('result object missing metadata for reordering bits')


This get raised when calling job.result() for all jobs that were run prior to the new metadata field, which seems quite recent. Should it at least return something, even though it could potentially miss some reorderings? Otherwise this PR is not immediately usable if one goes searching for past jobs.

Ok, we can return and log warning.

diego-plan9 · 2018-06-11T10:14:52Z

@ewinston I think this requires an update to IBMQuantumExperience 1.9.2 (should go in requirements.txt)

Yes - actually addressed in #558!

ajavadia · 2018-06-12T20:11:28Z

Can we provide an easy method to get the date/time of execution of a job? I think that's useful for filtering, when someone gets a long list of past executions. It could go in the status for DONE jobs, or it could be a separate method of IBMQJob.

ajavadia · 2018-06-12T20:12:08Z

Other than that I think everything looks pretty great!

ajavadia · 2018-06-13T02:54:09Z

qiskit/backends/ibmq/ibmqbackend.py

+            status = JobStatus[status]
+        while len(job_list) < limit or len(job_info_list) < limit:
+            base_index += limit
+            job_info_list = self._api.get_jobs(limit=limit, skip=base_index)


Why do we have to make repeated API calls here? Seems like the API should be able to return all the jobs at once. If it is not, perhaps we should update the API.

ajavadia · 2018-06-13T05:04:15Z

qiskit/backends/ibmq/ibmqbackend.py

+        backend_name = self.configuration['name']
+        job_list = []
+        base_index = skip
+        job_info_list = self._api.get_jobs(limit=limit, skip=base_index,


a bit of code restructuring:
I think the 2 calls to the API here can be made just one. Place it in the first line of the while loop. Then if all the jobs are retrieved in the first call, the while loop terminates. The way it is written right now, at least 2 calls are made every time.

ajavadia · 2018-06-13T15:08:56Z

@ewinston this looks good to merge to me. Can you please confirm that the TODO section of the original PR description is done now?

ewinston · 2018-06-13T15:23:20Z

@ajavadia yes I believe these are done now.

ajavadia · 2018-06-15T05:05:47Z

The new speed improvement is really good. Mine went from 13 secs to 0.9 secs

atilag · 2018-06-15T09:40:39Z

qiskit/backends/ibmq/ibmqjob.py

+        job_instance._status_msg = None
+        job_instance._cancelled = False
+        job_instance._is_device = is_device
+        job_instance._from_api = True


Is this flag being used?

removed, thanks

atilag · 2018-06-15T09:45:31Z

qiskit/backends/ibmq/ibmqjob.py

        # pylint: disable=arguments-differ
        while self._status == JobStatus.INITIALIZING:
+            if self._future_submit.exception():
+                raise IBMQJobError('error submitting job: {}'.format(
+                    repr(self._future_submit.exception())))
            time.sleep(0.1)


I would encourage to at least change these loops for the ._wait_for_submmiting() method.

Since this seems like a performance improvement but it risks breaking things perhaps I can leave this for a separate PR?

atilag · 2018-06-15T09:47:09Z

qiskit/backends/ibmq/ibmqjob.py

@@ -97,12 +158,12 @@ def cancel(self):
    @property
    def status(self):
        if self._status == JobStatus.INITIALIZING:


With the _wait_for_submitting() there's no need for this check as well

atilag · 2018-06-15T09:48:08Z

qiskit/backends/ibmq/ibmqjob.py

-        return self._job_id
+        while not self._id:
+            # job is initializing and hasn't gotten a id yet.
+            time.sleep(0.1)


_wait_for_submitting() :)

atilag · 2018-06-15T09:54:03Z

qiskit/backends/ibmq/ibmqjob.py

-        while not (self.done or self.cancelled or self.exception):
-            if timeout is not None and timer >= timeout:
-                job_result = {'job_id': job_id, 'status': 'ERROR',
+        # qobj = self._q_job.qobj


nit: Remove commented code. There's no need for this having git :)

atilag · 2018-06-15T09:59:41Z

qiskit/backends/ibmq/ibmqjob.py

-            logger.info('status = %s (%d seconds)', api_result['status'], timer)
-            api_result = self._api.get_job(job_id)
+            logger.info('status = %s (%d seconds)', api_result['status'],
+                        elapsed_time)

            if 'status' not in api_result:


Ok, seems like there is the possibility of a job having the status of CANCELLED, so it has not results. We should be treating this status in the conditionals as well.

atilag · 2018-06-15T10:05:27Z

qiskit/backends/ibmq/ibmqjob.py

+        #             id)
+        api_result = None
+        start_time = time.time()
+        while not (self.done or self.cancelled or self.exception or


This is sending at least one more request.

atilag · 2018-06-15T10:08:14Z

qiskit/backends/ibmq/ibmqjob.py

                              'result': api_result['status']}
-                return Result(job_result, qobj)
+                return Result(job_result)

        if self.cancelled:


This will send another request to the server, I would just check against api_result['status']. There's no way a job can be canceled by the user while is calling result, as this is a blocking call.

atilag · 2018-06-15T10:09:15Z

qiskit/backends/ibmq/ibmqjob.py

+                      'status': api_result['status'],
+                      'used_credits': api_result.get('usedCredits'),
+                      'result': job_result_list}
+        # logger.info('Got a result for qobj: %s from remote backend %s with job id: %s',


nit: Remove commented code (or leave the trace there :))

atilag · 2018-06-15T10:28:51Z

test/python/test_ibmqjob.py

+            self.assertTrue(isinstance(job.id, str))
+        self.log.info('time to get job statuses: %0.3f s', time.time() - start_time)
+
+    def test_retrieve_job(self):


I would make a test for a Job with multiple circuits.

test_run_simulator does this

This bug would cause the method to miss first "limit" number of jobs.

Previously retrieving jobs without metadata would raise an exception. Now a warning is issued. Also, the property "creationDate" was added to ibmq jobs.

Previously backend.jobs() more api calls than it needed. Also, ibmqjob.job_id is now ibmqjob.id

@delapuente

This commit incorporates improvements from PR Qiskit#515 by @delapuente to handling job status such that it makes fewer api calls. Co-authored-by: [email protected]

for status of RUNNING or QUEUED or DONE the status_msg would indicate "initializing"

when calling job.status for a server with a suspended queue the command would hang.

atilag · 2018-06-18T10:51:17Z

qiskit/backends/ibmq/ibmqbackend.py

+        if isinstance(status, str):
+            status = JobStatus[status]
+        while len(job_list) < limit or len(job_info_list) < limit:
+            job_info_list = self._api.get_jobs(limit=limit, skip=base_index,


With the latest version of IBMQuantumExperience package (1.9.4), we can pass a filter to the get_jobs(), so we can filter by backend name and job status, making this whole algorithm way simpler and reducing the potential huge number of requests to 1.

That's great! Thanks for getting that through.

database filtering was introduced in API 1.9.4, which has much better performance, however it doesn't seem to filter on status properly for real devices.

atilag · 2018-06-19T21:10:40Z

After talking with @ewinston we are going to leave jobs() filtering for a future release, when the related server bug is fixed. In the mean time, we will only filter jobs by backend name.

* add jobs() method to retrieve all jobs from a backend * job cancellation improvements for hubs * add retrieve_job() method for ibmq backends * add circuit names to non-qobj ibmq job submission * remove qobj from Result * add temporary hack for bit reordering * allow retrieving old jobs without metadata * property "creationDate" was added to ibmq jobs.

ewinston requested review from ajavadia, atilag, diego-plan9, ismaelfaro and jaygambetta as code owners May 24, 2018 10:22

ewinston force-pushed the backend_jobs branch from 19bf2eb to 2328076 Compare May 30, 2018 19:42

ewinston changed the title ~~[WIP] get jobs from IBM Q backend~~ get jobs from IBM Q backend Jun 1, 2018

diego-plan9 mentioned this pull request Jun 6, 2018

Complete the Jobs implementation #539

Closed

5 tasks

diego-plan9 added this to the 0.5.x milestone Jun 6, 2018

ewinston force-pushed the backend_jobs branch from 54348b8 to 70b5cd2 Compare June 6, 2018 18:59

ajavadia reviewed Jun 10, 2018

View reviewed changes

ewinston force-pushed the backend_jobs branch 2 times, most recently from 8caca6e to 7ce53ea Compare June 12, 2018 19:31

ajavadia previously approved these changes Jun 12, 2018

View reviewed changes

ajavadia reviewed Jun 13, 2018

View reviewed changes

ewinston dismissed ajavadia’s stale review via f98e4ab June 13, 2018 04:16

ajavadia reviewed Jun 13, 2018

View reviewed changes

ajavadia previously approved these changes Jun 13, 2018

View reviewed changes

ajavadia added the priority: high label Jun 14, 2018

ewinston dismissed ajavadia’s stale review via 855d872 June 14, 2018 22:41

atilag reviewed Jun 15, 2018

View reviewed changes

ewinston added 2 commits June 15, 2018 15:41

job cancellation improvements for hubs

bcac889

add alternate ibmqjob init method; from_api()

cd825f4

ewinston added 8 commits June 15, 2018 15:41

fix bug in backend.jobs

0afb90d

This bug would cause the method to miss first "limit" number of jobs.

allow retrieving old jobs without metadata

7516e7a

Previously retrieving jobs without metadata would raise an exception. Now a warning is issued. Also, the property "creationDate" was added to ibmq jobs.

minor improvement to backend.jobs

674af67

linting

2b79d6b

speed up backend.jobs(). job_id->id

0cfd4cb

Previously backend.jobs() more api calls than it needed. Also, ibmqjob.job_id is now ibmqjob.id

linting

00cd596

incorporating some suggestions from @atilag

d7d1a4f

improvements to ibmqjob.status

10d0bc2

This commit incorporates improvements from PR Qiskit#515 by @delapuente to handling job status such that it makes fewer api calls. Co-authored-by: [email protected]

ewinston force-pushed the backend_jobs branch from 7bc7c87 to 10d0bc2 Compare June 15, 2018 19:48

ewinston added 5 commits June 15, 2018 15:58

linting

2e72457

linting

49edb41

linting

8581dcb

update job status_msg

11d859b

for status of RUNNING or QUEUED or DONE the status_msg would indicate "initializing"

fix hanging job.status on server error

a426d1b

when calling job.status for a server with a suspended queue the command would hang.

atilag suggested changes Jun 18, 2018

View reviewed changes

atilag mentioned this pull request Jun 18, 2018

Adding test suite for testing IBMQJob in isolation. #515

Merged

6 tasks

remove job status filtering

c2d1033

database filtering was introduced in API 1.9.4, which has much better performance, however it doesn't seem to filter on status properly for real devices.

ewinston force-pushed the backend_jobs branch from acdf2cd to c2d1033 Compare June 18, 2018 18:44

linting

cee799c

ewinston removed this from the 0.6 milestone Jun 19, 2018

Merge branch 'master' into backend_jobs

13274e6

atilag requested a review from delapuente as a code owner June 19, 2018 21:11

atilag approved these changes Jun 19, 2018

View reviewed changes

diego-plan9 added this to the 0.6 milestone Jun 20, 2018

atilag merged commit 572db84 into Qiskit:master Jun 20, 2018

ajavadia mentioned this pull request Jun 21, 2018

advanced IBMQBackend.jobs() filtering #585

Merged

ajavadia mentioned this pull request Jul 23, 2018

result.get_ran_qasm() not working #671

Closed

ewinston deleted the backend_jobs branch April 6, 2021 16:27

get jobs from IBM Q backend #501

get jobs from IBM Q backend #501

Conversation

ewinston commented May 24, 2018 • edited Loading

Description

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Checklist:

ajavadia commented Jun 10, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

diego-plan9 commented Jun 11, 2018

ajavadia commented Jun 12, 2018

ajavadia commented Jun 12, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ajavadia commented Jun 13, 2018

ewinston commented Jun 13, 2018

ajavadia commented Jun 15, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atilag commented Jun 19, 2018

ewinston commented May 24, 2018 •

edited

Loading