Tuesday, 2016-11-29

*** bobh_ has joined #openstack-mistral00:03
*** catintheroof has joined #openstack-mistral00:08
*** thrash is now known as thrash|g0ne01:32
*** bobh_ has quit IRC02:01
*** catintheroof has quit IRC02:16
*** catintheroof has joined #openstack-mistral02:16
*** catintheroof has quit IRC02:20
*** sharatss has quit IRC03:23
*** sharatss has joined #openstack-mistral03:24
*** catintheroof has joined #openstack-mistral03:25
*** catinthe_ has joined #openstack-mistral04:23
*** catintheroof has quit IRC04:26
*** catinthe_ has quit IRC04:26
*** catintheroof has joined #openstack-mistral04:27
*** catintheroof has quit IRC04:27
*** sharatss has quit IRC04:50
*** jrist has quit IRC05:03
*** hparekh has joined #openstack-mistral05:12
*** sharatss has joined #openstack-mistral05:14
openstackgerritRenat Akhmerov proposed openstack/mistral: Fix launch process of Mistral components  https://review.openstack.org/40239205:25
*** ist has joined #openstack-mistral05:44
openstackgerritJizhaoxuan proposed openstack/mistral: changes get next execution time from local time zone to UTC timezone  https://review.openstack.org/40367105:57
*** sharatss has quit IRC06:08
*** jaosorior has joined #openstack-mistral06:22
istHi all, I just would like to raise a new topic/feature request about action and workflow input validation against a given json schema.06:31
istWe have couple of custom actions and provided workflows for our customers, and we have already defined some json schema to describe the parameter lists of given workflows/actions06:33
rakhmerovok06:34
rakhmerovist: can you share more details?06:34
istAnd it would be nice if mistral could verify these schema somhow, before it starts the execution of given workflow/action06:34
rakhmerovist: you mean a schema for workflow input parameters?06:34
rakhmerovok, I see06:35
istCurrently I have no good idea about solution...06:35
istyes06:35
rakhmerovhm... let me think06:35
istfor actions it is possible to verify the input in __init__06:35
rakhmerovso, I'm not sure I understand on 100%06:36
istbut may a centralized verification would be nice for actions also06:36
rakhmerovwhat we have now is a check what parameters are missing and what redundant06:36
rakhmerovwhen we start a WF06:36
istSo we have workflows with complex input, and we would like to create a pre-condition like verification for their parameter structure06:37
*** sharatss has joined #openstack-mistral06:37
rakhmerove.g. if you you have a WF with input params A, B and C but you pass A, D, C, G Mistral will say that D and G are unknown param and B is missing06:37
rakhmerovist: ok, so you're talking about more complex params with their own structure06:38
rakhmerovnot just a plain list of params06:38
istyes I know, but if a workflow has got a deep structure as parameter only the root elements of parameter objects are checked currently06:38
rakhmerovI see06:38
rakhmerovyes06:38
rakhmerovhm.. I wonder how it could be described in workflow text06:39
rakhmerovgenerally I think it's a good thing06:39
istPlus if we could solve the validation of workflow parameters, we basically solves the problem of failing yaql/jinja expressins during workflow run06:39
rakhmerovyes, at least partially06:40
rakhmerovist: can you please file a BP for that with some basic explanations and the note that we need a spec?06:40
istAs we discussed earlier we had some problem with yaql's in task inputs, which causes on-error branches are not executed06:41
rakhmerovyep06:41
istOk, I will write a short summary about that06:42
rakhmerovyes, thanks06:43
rakhmerovgenerally, I'm ok with the idea itself06:43
rakhmerovddeja: hi06:43
ist+ one more challenge: Later we would need create new version from actions/workflows with modified schema. And we shall keep backward compatibility also, so we have to deal with multiple versions from same action/workflow...06:48
istnew name for modified actions is the easiest solution of course, but it is the we could do?06:48
*** jaosorior has quit IRC07:07
*** jaosorior has joined #openstack-mistral07:08
ddejarakhmerov: hello07:38
rakhmerovddeja: pretty often when I run tests from PyCharm Mistral can't find 'transport_url' option in DEFAULT group07:39
rakhmerovany ideas what is wrong?07:39
openstackgerritMerged openstack/python-mistralclient: Make python mistralclient readme better  https://review.openstack.org/40379807:39
ddejarakhmerov: but only from pycharm?07:40
ddejaI mean, it works from 'pure' tox?07:40
rakhmerovhm.. didn't try tox :)07:40
ddejaok07:41
ddejaand one more thing: it's often, so sometimes it works?07:41
rakhmerovfor some tests07:41
rakhmerovalways07:41
ddejaok, it's something07:41
rakhmerovwhat I don't understand is IMO this option should be defined when importing oslo.config07:41
ddejacan you give me a name of such test?07:42
rakhmerovbut it doesn't happen07:42
rakhmerovsecond, looking..07:42
ddejayes, this option is a default option for oslo, so it should just be there07:42
rakhmerovyes, that's what I thought07:42
rakhmerovgosh, I just saw it an hour ago.. But can't find it07:44
rakhmerovsomething with actions07:44
ddejaok, I see07:44
ddejahm, to be honest I didn't run test for a while, let me check if it also happens in my env07:45
ddejamaybe this option got deleted on some version of oslo...07:45
rakhmerovyeah, found it07:46
rakhmerovtest_trigger_service.test_single_execution_with_multiple_processes07:46
rakhmerovtry it please07:46
rakhmerovtry it individually07:46
ddejaok07:47
rakhmerovI guess something is missing in test initialization but I don't see what07:47
ddejaI'll look on it and let you know when I found something07:47
rakhmerovok, thanks a lot07:48
ddejarakhmerov: on my env every test works fine using tox...07:54
rakhmerovhow do you run this test?07:54
rakhmerovwith what command?07:55
ddejatox -e py2707:55
*** ist_ has joined #openstack-mistral07:56
ddejarakhmerov: could you try run it with tox, so we can be sure that there is a problem with tests, not with pycharm + tests?07:57
rakhmerovnow, the problem occurs when if I run a test individually07:59
rakhmerovif I run them all then all is fine07:59
ddejaoh07:59
rakhmerovyes07:59
ddejaok, let my try just this one07:59
rakhmerovit's not a serious issue but it's really annoying during development when I just need to run one test08:00
ddejarakhmerov: also running only this one works http://paste.openstack.org/show/590756/08:03
rakhmerovvia tox?08:04
ddejayes08:05
ddejahere's the command tox -e py27 -- mistral.tests.unit.services.test_trigger_service.TriggerServiceV2Test.test_single_execution_with_multiple_processes08:05
rakhmerovlet me try08:06
*** shardy has joined #openstack-mistral08:07
rakhmerovhm.. it's recreating py27 env08:08
ddejayes08:09
rakhmerovI guess the problem is in import order somewhere08:10
rakhmerovwith oslo.messaging08:10
rakhmerovmaybe it's imported too late somewhere08:10
ddejawait, I remember that once I was fixing similar issue08:10
rakhmerov:)08:10
ddejalet me go trought my patch-sets08:10
rakhmerovok08:10
*** ist_ has quit IRC08:11
rakhmerovand btw, I confused you. This option should be registered by oslo.messaging, not oslo.config08:11
rakhmerovmaybe we should load this option manually somewhere like "CONF.import_opt('rpc_response_timeout', 'mistral.config')"08:13
*** jpich has joined #openstack-mistral08:16
rakhmerovmy tox test is still running..08:22
ddejarakhmerov: hm, so 2 things08:24
ddeja1) IMO it is something wrong with your env, since it is passing on mine and on devstack08:24
ddeja2) you can take a look on this https://review.openstack.org/#/c/344152/08:25
ddejamaybe applaying this patch on top of the failing test would fix it?08:25
rakhmerovok08:26
rakhmerovyeah, it works for me if I use tox08:26
ddejaoh...08:27
rakhmerovso the problem only with PyCharm08:27
rakhmerovok, no worries, I'll see what I can do08:27
rakhmerovthanks for your help08:27
ddejahm, so we should take a look what are the differences beetwen tox and PyCharm08:29
ddejarakhmerov: no problem :)08:29
ddejaI have a meeting, I'll be back in 0.5 hour08:29
rakhmerovyes08:29
rakhmerovok08:29
openstackgerritRenat Akhmerov proposed openstack/mistral: Fix launch process of Mistral components  https://review.openstack.org/40239208:29
d0ugalHey08:52
d0ugalI'd also like to backport this to newton if people think it is okay: https://review.openstack.org/#/c/404087/08:52
openstackgerritMerged openstack/mistral: Updated the retries_remain statement  https://review.openstack.org/38361709:07
openstackgerritMerged openstack/mistral: Show team and repo badges on README  https://review.openstack.org/40300809:15
openstackgerritMerged openstack/mistral: Initial commit for mistral-i18n support  https://review.openstack.org/40363909:15
openstackgerritRenat Akhmerov proposed openstack/mistral: Fix launch process of Mistral components  https://review.openstack.org/40239209:23
d0ugalFun, we have another way to get a workflow stuck in the running state09:34
* d0ugal attempts a minimal reprodce of this one09:34
d0ugalAnyone seen an error like this? http://paste.openstack.org/show/590760/09:42
*** openstackgerrit has quit IRC09:48
*** openstackgerrit has joined #openstack-mistral09:49
openstackgerritRenat Akhmerov proposed openstack/mistral: Fix launch process of Mistral components  https://review.openstack.org/40239209:50
*** brunograz has quit IRC09:50
d0ugalSo, if custom actions return invalid unicode, then workflows get stuck in the RUNNING state09:55
* d0ugal opens another bug09:56
rakhmerovd0ugal: ok, no objections on the backport09:59
d0ugalrakhmerov: thanks10:01
rakhmerovd0ugal: as far as the error, no, I haven't seen it10:01
rakhmerovd0ugal: such errors make me think that something may be wrong with how we handle errors in engine10:01
d0ugalrakhmerov: https://bugs.launchpad.net/mistral/+bug/164564910:02
openstackLaunchpad bug 1645649 in Mistral "When custom actions return invalid unicode it causes an error in mistral and the workflow gets stuck in the RUNNING state" [High,Confirmed] - Assigned to Dougal Matthews (d0ugal)10:02
d0ugalrakhmerov: in this case, the error is in the executor log10:02
d0ugaloops10:02
d0ugalI pasted the wrong traceback, let me get the correct one10:02
d0ugalsorry10:03
rakhmerovmy assumption is: whatever happens with an action must be handled properly so that we see updated states of all objects10:03
rakhmerovin other words: we shouldn't be just swallowing any errors10:03
d0ugalI updated the bug, but here it is too (since launchpad formatting is terrible): http://paste.openstack.org/show/590764/10:04
rakhmerovthey all should be reflected in persistent state of corresponding objects10:04
d0ugalAgreed10:04
rakhmerovI would like to look at this one10:04
rakhmerovif you don't mind10:04
d0ugalrakhmerov: sure10:04
rakhmerovbecause I recently did a lot on error handling10:04
d0ugalrakhmerov: suits me :)10:04
d0ugalOur bad unicode came from a corrupt file in swift, but it should be fairly easy to reproduce I think10:05
rakhmerovyeah10:06
rakhmerovI'll do it10:06
rakhmerovsuggestion: let's assign "critical" priority to bugs like this10:07
rakhmerovwhich lead to WFs stuck in RUNNING state10:07
rakhmerovd0ugal: it's very cool that you keep finding them10:08
rakhmerovmost of them are easy to fix but they improve stability significantly10:09
d0ugalrakhmerov: I only found one. credit for finding this one should go to jpich :)10:09
rakhmerovjpich: thanks!10:09
rakhmerov:)10:09
jpichYou're welcome :-)10:09
* jpich breaking things10:09
d0ugaljpich: including files ;)10:09
jpichAll the things!10:10
rakhmerovd0ugal: btw, seems like I was able to make all the tests pass for my big patch: https://review.openstack.org/#/c/402392/10:10
d0ugalrakhmerov: oh, nice. Is this where you had problems with sqlite?10:10
rakhmerovnot exactly sqlite10:10
rakhmerovwith threads10:10
rakhmerovI was asking about sqlite for a different reason )10:10
rakhmerovd0ugal, ddeja, hparekh: guys, please review https://review.openstack.org/#/c/402392/10:11
rakhmerovit is big but I couldn't really make it smaller because many things were entangled too much10:11
d0ugalHopefully it is less entagled as a result :)10:11
d0ugalI'll try and review now.10:12
rakhmerovindeed!10:12
rakhmerovthat was the purpose10:12
rakhmerovtesting is not enough now, I'm planning to add tests for all new classes10:12
rakhmerovindirectly they are tested though10:12
rakhmerovI just don't wanna make this patch even bigger10:12
d0ugalsure10:13
d0ugaldoes this have any user impact?10:13
rakhmerovddeja, hparekh: the patch is tricky, I couldn't put all the info in the commit message, ask questions as you review10:13
rakhmerovd0ugal: not too big except they won't see warnings from oslo.messaging anymore10:13
d0ugalI had wondered about that warning but ignored it :-D10:14
rakhmerovand there might have been some issues with messaging on the first seconds after launch10:14
* ddeja is reviwing https://review.openstack.org/#/c/402392/10:14
rakhmerovd0ugal: yeah, that warning was the initial driver for this whole work10:14
rakhmerovwhile investigating it I just realized that it's all messed up very much10:15
rakhmerovI couldn't hold myself from refactoring it, sorry10:15
rakhmerovddeja: btw, please note that I left some comments related to RPC in this patch10:17
rakhmerovthis is something that I think we can improve10:17
rakhmerovlike having get_transport() method in rpc.py10:17
ddejarakhmerov: OK, I'l pay extra attention to them10:18
rakhmerovok10:19
openstackgerritJizhaoxuan proposed openstack/mistral: Add PyV8 into requirement.txt  https://review.openstack.org/40413510:33
d0ugalrakhmerov: I think the patch looks pretty good10:36
rakhmerovI like it too!10:36
rakhmerov:))10:36
rakhmerovDawid usually finds a lot of stuff :)10:36
d0ugalhah, that is probably because he understands this more than me10:37
d0ugalI feel like I just understand it10:37
rakhmerovthat's good10:37
d0ugalso we are going to have a mistral.service and mistral.services now10:39
d0ugalthat strikes me as a bit confusing :)10:39
rakhmerovyes, good point10:39
rakhmerovI think we need to rename "services"10:39
rakhmerovhonestly, I never liked it10:39
d0ugalthey don't really feel like services10:39
rakhmerovessentially, all that's inside services has nothing to do with what we usually mean by services10:40
rakhmerovyes, exactly10:40
d0ugallol10:40
rakhmerovso, it's a temporary clash, we'll take care of it10:40
d0ugalsure10:40
d0ugalrakhmerov: I spotted one print statement that probably needs removed, but otherwise I think it looks good. I think you said you want to update the tests so I'll take another look then10:42
d0ugalIt makes me happy that we have TripleO CI testing changes like this :)10:42
rakhmerovwell, I was planning to write more test in a different patch actually10:42
d0ugalah10:43
rakhmerovthat print statement is intentional10:43
rakhmerovthe idea is to always print that the service is start to std.out10:44
rakhmerovso that it works with any logging settings10:44
rakhmerovdon't know, maybe it's not so good but it's intentional10:44
openstackgerritSharat Sharma proposed openstack/mistral: Added releasenote for retry policy update  https://review.openstack.org/40414210:44
rakhmerovd0ugal: I replied to your comment in the review10:46
rakhmerovbut! I found really a couple of prints that are not intentional :)10:46
d0ugalrakhmerov: hah :)10:47
rakhmerovin test_run_action10:47
d0ugalrakhmerov: Wouldn't logging make more sense?10:47
rakhmerovwill resend it in a min10:47
d0ugalThe LOG isn't used in that file at all10:47
rakhmerovd0ugal: aah, don't know. As a user, when I launch Mistral I'd like to see all the basic stuff right on the console10:47
rakhmerovlike service A, B and C started10:48
d0ugalSure, fair enough10:48
d0ugalThat's fine10:48
openstackgerritRenat Akhmerov proposed openstack/mistral: Fix launch process of Mistral components  https://review.openstack.org/40239210:48
*** dtantsur|afk is now known as dtantsur11:10
*** hparekh has quit IRC11:11
*** sharatss has quit IRC11:24
*** sharatss has joined #openstack-mistral11:24
ddejarakhmerov: not a lot of stuff, only 8 comments ;)11:32
ddejaand half of it are questions, I think11:33
*** brunograz has joined #openstack-mistral12:06
*** thrash|g0ne is now known as thrash12:23
sharatssrakhmerov, d0ugal anybody there?12:30
sharatssddeja, hi12:31
dtantsurhey folks! may I run one idea through you re microversioning?12:34
dtantsurwhat if we bump the default microversion certain services use?12:35
dtantsurlike, do we really expect Mistral Ocata to work with Ironic Juno?12:35
dtantsurd0ugal, ^^^12:35
d0ugaldtantsur: good question.12:37
*** catinthe_ has joined #openstack-mistral12:37
d0ugaldtantsur: it does make sense to me to default Mistral to work with the same version of other services12:37
d0ugalso mistral ocata should default to ironic ocata12:37
dtantsurI'd defer to Ironic Newton for smoother upgrade, but yeah12:37
dtantsurI guess I'll put up a patch and see what folks say12:38
d0ugaldtantsur: sounds good12:38
ddejahi sharatss12:42
ddejasorry, I was on lunch12:42
*** shardy is now known as shardy_lunch12:47
*** ist has quit IRC12:51
*** ist has joined #openstack-mistral12:53
*** dprince has joined #openstack-mistral12:54
sharatssddeja, i was creating client docs12:57
sharatssddeja, i want to point to the v2 api docs that are12:57
sharatssddeja, that is there in mistral repo now. can i directly provide a link to the html file of v2 api or should i copy the same rst into client repo and create a local link?12:59
*** weshay_pto is now known as weshay13:03
*** bobh_ has joined #openstack-mistral13:04
openstackgerritDmitry Tantsur proposed openstack/mistral: Bump Ironic API version to 1.22 when creating the Ironic client  https://review.openstack.org/40420313:06
dtantsurd0ugal and others ^^^13:06
ddejasharatss: IMO you can provide a link to the html file13:15
ddejawe should not have the same doc in two places13:16
sharatssddeja, thanks13:21
*** jrist has joined #openstack-mistral13:23
*** bobh_ has quit IRC13:28
*** shardy_lunch is now known as shardy13:28
openstackgerritfengchaoyang proposed openstack/mistral: Fix two errors in YAML example  https://review.openstack.org/40422913:56
zanebquick question: how do you access the environment from yaql?14:37
zanebapparently it's not $.env()14:38
zaneb:)14:38
openstackgerritMichal Gershenzon proposed openstack/python-mistralclient: Make python mistralclient readme better  https://review.openstack.org/40424914:45
d0ugalzaneb: I think it is just env() - it is a global function rather than part of $.14:49
zanebah, thanks d0ugal14:49
_mgershend0ugal: +115:01
*** bobh_ has joined #openstack-mistral15:04
*** jaosorior has quit IRC15:07
*** jaosorior has joined #openstack-mistral15:08
*** ist has quit IRC15:08
*** chlong has joined #openstack-mistral15:27
openstackgerritMerged openstack/mistral: Some spelling errors  https://review.openstack.org/40349715:45
*** hrybacki is now known as hrybacki|mtg15:50
*** jaosorior has quit IRC16:05
*** chlong has quit IRC16:07
*** chlong has joined #openstack-mistral16:20
*** thrash is now known as thrash|biab16:28
openstackgerritfengchaoyang proposed openstack/python-mistralclient: Move json.loads() method to utils.py, and use "with" to deal with file objects  https://review.openstack.org/40433616:41
*** hrybacki|mtg is now known as hrybacki16:45
*** thrash|biab is now known as thrash16:54
*** dtantsur is now known as dtantsur|afk17:04
*** chlong has quit IRC17:10
*** jpich has quit IRC17:21
*** chlong has joined #openstack-mistral17:24
*** dprince has quit IRC17:50
*** dprince has joined #openstack-mistral17:50
*** dprince has quit IRC18:25
*** dprince has joined #openstack-mistral18:26
*** shardy has quit IRC18:34
*** bobh_ has quit IRC20:34
*** catinthe_ has quit IRC20:35
*** catintheroof has joined #openstack-mistral21:34
*** catintheroof has quit IRC21:38
*** rbrady is now known as rbrady-afk21:44
*** catintheroof has joined #openstack-mistral21:47
*** jrist has quit IRC22:08
*** chlong has quit IRC22:12
*** dprince has quit IRC22:35
*** sharatss has quit IRC22:59
*** sharatss has joined #openstack-mistral22:59
*** catinthe_ has joined #openstack-mistral23:03
*** catintheroof has quit IRC23:06
*** bobh has joined #openstack-mistral23:13

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!