Create Workitem - HTTP 500

[Mershl]

Hi dixudx,

Thanks for this project. You already helped me a lot. The last piece that is currently not working for me is creating RTC workitems.

It results in a HTTP 500 error:

End of log (HTTP error 500):
DEBUG requests.packages.urllib3.connectionpool: https://rtchost:443 "POST /ccm/oslc/contexts/_qi848CJaEeSy9NmiE0zU4Q/workitems/task HTTP/1.1" 500 184
ERROR client.RTCClient: Failed POST request at <https://rtchost/ccm/oslc/contexts/_qi848CJaEeSy9NmiE0zU4Q/workitems/task> with response: <?xml version="1.0" encoding="UTF-8"?><oslc_cm:error xmlns:oslc_cm="http://open-services.net/xmlns/cm/1.0/">
<oslc_cm:status>500</oslc_cm:status>
<oslc_cm:message/>
</oslc_cm:error>

Trying to access .../shapes/workitems/task manually results in the following. I think the OSLC format of the workitem creation is no longer supported. Is this possible?

<oslc_cm:error>
<oslc_cm:status>406</oslc_cm:status>
<oslc_cm:message>**OSLC Version '1.0' is not supported.**</oslc_cm:message>
</oslc_cm:error>

[dixudx]

@Mershl Thanks for using this library.

Probably. Which version is your rtc jazz server. Currently this library has been tested against 5.0.1, 5.0.2 and 6.0.

[Mershl]

We have a RTC Jazz 6.0.2 server in use.

[dixudx]

@Mershl Currently I don't have such 6.0.2 environment in handy. So would you please help fix it. Thanks.

[Feva]

@dixudx As a side point to this and #87 we are running 6.0.3

[Mershl]

I've tested some fixes but couldn't successfully create a workitem with 6.0.2 yet. This is the complete log with an unmodified rtcclient: https://gist.github.com/Mershl/1f70e294e107f67238bef45897a7684d

[dixudx]

@Mershl Thx. I will take a closer look. Sorry for the inconveniences.

[Mershl]

Adding

        headers['Accept'] = 'application/rdf+xml'
        headers['OSLC-Core-Version'] = '2.0'
        headers['Content-Type'] = 'text/xml'

to L#1119 in [rtcclient/client.py].
Results in two outcomes:

1. HTTP 500, showing a unset expire date in the cookie:
   WASReqURL=""; Expires=Thu, 01-Dec-94 16:00:00 GMT; <...>

2. HTTP 403, showing missing attribute 'Summary':

<rdf:Description rdf:nodeID="A0">
    <oslc:message>'Save Work Item' failed. Preconditions have not been met: The 'Summary' attribute needs to be set (work item &lt;11:42:22&gt;).</oslc:message>
    <oslc:statusCode>403</oslc:statusCode>
    <rdf:type rdf:resource="http://open-services.net/ns/core#Error"/>
  </rdf:Description>

Out of 6 consecutive runs. 4 result in expired cookie and 2 in missing attribute.

[dixudx]

@Mershl Great! Good progress.

I think

headers['Accept'] = 'application/rdf+xml'
headers['OSLC-Core-Version'] = '2.0'

is the key that matters. From your past log, oslc-core-version 1.0 is not supported any more in 6.0.3.

We can try to add a new attribute Summary when posting a new workitem.

[Mershl]

Removing the modification of Content-Type results in

<rdf:Description rdf:nodeID="A0">
    <oslc:message>Content type 'application/x-oslc-cm-change-request+xml' is not supported.</oslc:message>
    <oslc:statusCode>415</oslc:statusCode>
    <rdf:type rdf:resource="http://open-services.net/ns/core#Error"/>
  </rdf:Description>

[dixudx]

@Mershl Thanks for your work. It helps me a lot.

Seems headers['Accept'] = 'application/rdf+xml' is also indispensable.

Can you successfully create a new workitem when appending a new attribute Summary.

[Mershl]

I use the template of a existing work item on the server. The template does not offer a Summary attribute (which is weird, as it is required), in which case the Summary argument inside kwargs of createWorkitem is discarded by renderFromWorkitem(copied_from, <...>). How would you add an attribute to an existing template?

[Mershl]

Based on the spec there is no Summary attribute: OSLC Spec.

[Mershl]

I think the problem lies within the XML header attributes.
Currently:

<oslc_cm:ChangeRequest 
    xmlns:oslc_cm="http://open-services.net/xmlns/cm/1.0/" 
    xmlns:calm="http://jazz.net/xmlns/prod/jazz/calm/1.0/" 
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" 
    xmlns:dc="http://purl.org/dc/terms/" 
    xmlns:oslc_pl="http://open-services.net/ns/pl#" 
    xmlns:rtc_cm="http://jazz.net/xmlns/prod/jazz/rtc/cm/1.0/">

[dixudx]

@Mershl Actually the Summary comes from dc:title.

[Mershl]

@dixudx You're right. But dc:title is specified in the POST request.

<dc:title>OSLC TestTitle</dc:title>

[dixudx]

Yes, it's quite weird. It works well on my previous environment and has been fully tested.

I wonder whether the oslc version does the trick.

[Mershl]

I've tested a number of combinations and additions for the XML header, but the best outcome is the The 'Summary' attribute needs to be set 403 failure. Do you have an idea what could cause this?

[dixudx]

@Mershl Haven't got any clues yet. Will try to find it out.

[dixudx]

403 means the request body is not accepted by the server, either the header or the attributes.

[dixudx]

@Mershl @Feva Please try this patch. I've already tested on my env. All works well.

Special Note: Please run pip install -r requirements.txt to install new package lxml.

[Mershl]

@dixudx Thanks for the quick patch.

Due to the etree import from lxml the RTCClient is no longer compatible to Python 2.7.

  File "<...>\python_deps\rtcclient\utils.py", line 7, in <module>
    from lxml import etree
ImportError: cannot import name etree

I have to support Python 2 due to other dependencies in my project. But I've tried it with 3.5.2 showing:

2017-06-02 08:05:57,408 DEBUG urllib3.connectionpool: https://<jazzhost>:443 "GET /ccm/oslc/workitems/305276 HTTP/1.1" 200 17127
2017-06-02 08:05:57,424 DEBUG template.Templater: Successfully remove field [calm:affectsExecutionResult] from the template
Traceback (most recent call last):
  <...>
  File "<...>\python_deps\rtcclient\client.py", line 1048, in createWorkitem
    **kwargs)
  File "<...>\python_deps\rtcclient\client.py", line 1151, in _checkMissingParamsFromWorkitem
    keep=keep)
  File "<...>\python_deps\rtcclient\client.py", line 880, in listFieldsFromWorkitem
    keep=keep)
  File "<...>\python_deps\rtcclient\template.py", line 184, in listFieldsFromWorkitem
    keep=keep)
  File "<...>\python_deps\rtcclient\template.py", line 274, in getTemplate
    self._remove_long_fields(wk_raw_data)
  File "<...>\python_deps\rtcclient\template.py", line 358, in _remove_long_fields
    for key in wk_raw_data.keys():
RuntimeError: OrderedDict mutated during iteration

[dixudx]

@Mershl I've tested on 2.7.13 and 3.3.6. lxml works well.

(venv_3.3.6) vagrant@vagrant:~$ python
Python 3.3.6 (default, May 25 2017, 14:23:22) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from lxml import etree
>>> exit()
(venv_3.3.6) vagrant@vagrant:~$ workon venv_2.7.13
(venv_2.7.13) vagrant@vagrant:~$ python
Python 2.7.13 (default, May 25 2017, 14:02:15) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from lxml import etree
>>> 

[dixudx]

@Mershl Yeah, on python 3.5, the runtime error did exist. Fixed in commit 430f525.

[Mershl]

I've installed the additional dependencies on Windows (sign) for lxml. The work item is now successfully created. Thank you very much for your support @dixudx.

[krishnavelu]

Hello dixudx,

Sorry for the late reply.
I've tried that commit. It give me an error 'str' object has no attribute 'decode'.
I'm using python 3.7.

But changing that function like this worked for me...
I was able to create work items.

def remove_empty_elements(docs):
    root = etree.fromstring(bytes(docs, 'utf-8'))
    for element in root.xpath("//*[not(node())]"):
        if "rdf:resource" not in str(etree.tostring(element)):
            element.getparent().remove(element)
 
    return etree.tostring(root)

Mostly all of the problems are solved in some commits here and there. But its not yet merged in development.
When will those get merged to development branch and when we can have 0.7.0 eventually...?

[dixudx]

Mostly all of the problems are solved in some commits here and there. But its not yet merged in development.

It is not yet merged in master branch.

When will those get merged to development branch and when we can have 0.7.0 eventually...?

I am so glad that this library is so widely used. And thanks for all of you to get this improved.

Please help try latest development branch to verify the newest fix works for you. 0.7.0 is coming soon.

If everything works well, I think 0.7.0 is ready to ship.

[krishnavelu]

Hello Dixudx,

Here is what I get, when I try to install development branch.

TypeError: environment can only contain strings

[dixudx]

@krishnavelu See my comment in your gist.

[krishnavelu]

Hello Dixudx,

Even with python3.5 I'm getting the same environment error.

See here
This might be due to the requirement 'pbr!=0.7,<1.0,>=0.6'
Can we not use the latest pbr 4.3.0?

Its just a guess, you may be the right person to answer.

Please note that I'm using windows 10. Which has new type of environment variable

[krishnavelu]

Hello Dixudx,

One more problem I face now for creating work item is, I'm not able to set a custom attribute.
I've created a custom attribute 'PvcsAcceptedBy' in RTC. And setting through api.
But I'm getting error.

'RTCClient' object has no attribute 'getPvcsAcceptedBy'.

Is custom attributes not supported? If not supported what modification should I do for adding custom attributes.

[dixudx]

This might be due to the requirement 'pbr!=0.7,<1.0,>=0.6'
Can we not use the latest pbr 4.3.0?

@krishnavelu You can give it a try. I don't have an available Windows environment.

'RTCClient' object has no attribute 'getPvcsAcceptedBy'.

Is custom attributes not supported? If not supported what modification should I do for adding custom attributes.

Yes, custom attributes are well supported. You can follow this createWorkitem API. Normally there are two ways to do this,

1. Uses a template. You can try to call getTemplate to get a template and make your own customizations there.
2. Create a new workitem by copying from another workitem. There is a parameter called copied_from in createWorkitem API

[krishnavelu]

Hello Dixudx,

I guess I'm using the copied_from one. But still I get the error

''RTCClient' object has no attribute 'getPvcsAcceptedBy''

'PvcsAcceptedBy' is the custom attribute.

The work Item gets created but, it doesn't set the value which is passed into createWorkItem. The value of that attribute gets copied from the original WI.

Here is the code I used to create WI.

wk = rtc.createWorkitem(item_type = "Story", title = "Dummy Story Title",
description = "Dummy Story description", projectarea_name = "project area name",
copied_from = "41048", filedAgainst = "filedAgainst",
plannedFor = "Release 1", ownedBy = ownedBy,
priority = "Unassigned", severity = 'Normal',
foundIn = 'Coding', PvcsAcceptedBy = "acceptedByPerson"
)

I really didn't understand the template concept even after reading the document multiple time. Also after getting the template how to modify it for custom attributes?

Could you please give me a example code for creating with custom attribute? What should be passed with **kwargs

[dixudx]

wk = rtc.createWorkitem(item_type = "Story", title = "Dummy Story Title",
description = "Dummy Story description", projectarea_name = "project area name",
copied_from = "41048", filedAgainst = "filedAgainst",
plannedFor = "Release 1", ownedBy = ownedBy,
priority = "Unassigned", severity = 'Normal',
foundIn = 'Coding', PvcsAcceptedBy = "acceptedByPerson"
)

@krishnavelu PvcsAcceptedBy is invalid. Here **kwargs are for args like filedAgainst, foundIn and etc, if you does not use a customized template.

replace_fields = [("rtc_cm:teamArea", "{{ teamArea }}"),
                  ("rtc_cm:ownedBy", "{{ ownedBy }}"),
                  ("rtc_cm:plannedFor", "{{ plannedFor }}"),
                  ("rtc_cm:foundIn", "{{ foundIn }}"),
                  ("oslc_cm:severity", "{{ severity }}"),
                  ("oslc_cm:priority", "{{ priority }}"),
                  ("rtc_cm:filedAgainst", "{{ filedAgainst }}")]

You can try to use getTemplate to save a template and insert some customized params, like pvcsAcceptedBy. Then you can use pvcsAcceptedBy when calling createWorkitem.

[krishnavelu]

Can I get some example code snippets for modifying the template.
I'm able to do getTemplate. But couldn't get any information about how to modify the custom attributes.

[dixudx]

@krishnavelu https://rtcclient.readthedocs.io/en/latest/client.html#rtcclient.client.RTCClient.createWorkitem

myclient.createWorkitem("workitem", "title name", description=None, projectarea_id="1234456", template=None, copied_from=23446, keep=False, foundIn="abc", filedAgainst="djlgjal", foundIn="dgag", mycustomized_attr="abc")

You just need to pass your template attribute in createWorkitem. But you need to make sure mycustomized_attr is supported or existed in your RTC environment.

[jacobbrozenick]

Any chance the development branch with these changes on it will get committed to master? I struggled with this exact error for almost 2 weeks before finally finding this thread, works great now! But I had to pull the development branch (which isn't the version in pypi).

[gunasekarjo]

@dixudx Hi I am facing some issue with create work item can u tell me which branch is completely fixed currently I am using development branch.

[Narendra0908]

@dixudx Thanks for this. Working now. I used:

myclient.getOwnedBy('[email protected]').url.split(urlquote('@'))[0]

So as to leave the standard code alone.

Got template working as well. Notice that palnnedFor was not working but added this in the template like other resources so all fine.

Hi @Feva can u please help to create a new workitem using createWorkitem( ) function?

[Narendra0908]

@dixudx can we connect some times because still i was not able to create the workitem using rtcclient. i need u r help very badly

[dixudx]

@Narendra0908 Feel free to send me an invitation via Google Meet. You could find my email address through the commits. Thanks.

[Narendra0908]

@dixudx how to update a work item?

[Narendra0908]

log.txt

[Narendra0908]

log.txt

[Narendra0908]

log.txt

[Narendra0908]

log.txt

[Narendra0908]

@dixudx when i tried to use the code that u send me via mail the following error occurs: ConnectionError: HTTPSConnectionPool(host='rb-alm-07-p.de.bosch.com', port=9443): Max retries exceeded with url: /jazz/authenticated/identity (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000001F48FED8760>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'))

[dixudx]

@Narendra0908 Please use your old way to initialize the RTCClient. And set a proper value for searchpath.

[Narendra0908]

will do the needful and remaining stuff we can decide in todays meeting scheduled @2

[dixudx]

We do have slack. Welcome to join with this invitation link https://join.slack.com/t/rtcclient/shared_invite/zt-269osdw4i-fygzMfIeS8s5mDMgNT2DSA

[Narendra0908]

I don't have the access for the slack link. Restricted from my company

[Narendra0908]

still can't access i will try using my personal pc for that since i am using my company network i don 't have access

[dixudx]

will do the needful and remaining stuff we can decide in todays meeting scheduled @2

@Narendra0908 Have you resolved the issues with my template? If so, we could cancel this meeting. Thanks.

[Narendra0908]

no the issue is not resolved
we need the meeting

[Feva]

@dixudx Thanks for this. Working now. I used:
myclient.getOwnedBy('[email protected]').url.split(urlquote('@'))[0]
So as to leave the standard code alone.
Got template working as well. Notice that palnnedFor was not working but added this in the template like other resources so all fine.

Hi @Feva can u please help to create a new workitem using createWorkitem( ) function?

Hi, this is all working now for me. I am running version 0.9.0 with the old authentication flag set.
We are on 7.0.2 RTC and all works fine.
I can create workitems.
I was running 0.6.0 as I used to get errors about planned fors etc not being correct but once moved to 0.9.0 and setting the old auth flag all works fine.
Thanks.