Monday, 2017-04-03

*** jkilpatr has quit IRC00:04
*** bobh has joined #openstack-mistral00:38
*** bobh has quit IRC00:44
*** Guest7322 has joined #openstack-mistral01:05
*** jamielennox|away is now known as jamielennox01:06
*** Guest7322 has quit IRC01:20
*** catintheroof has joined #openstack-mistral02:55
*** catintheroof has quit IRC04:10
rakhmerovtherve: hi05:42
rakhmerovyou here?05:42
rakhmerovtherve, d0ugal: just want to check if you got anything you on that SQLA issue05:43
rakhmerovI saw the patch from therve but I don't quite understand it. Is this a bug in SQLA after all or no?05:44
rakhmerovtherve: so, you just added this field in each subclass of Execution which seems strange to me05:44
rakhmerovonce you online please ping me05:45
rakhmerovtherve: also, did you have a chance to talk to Mike?05:45
*** sharatss has joined #openstack-mistral06:04
*** sharatss has quit IRC06:17
*** bobh has joined #openstack-mistral06:37
*** bobh has quit IRC06:42
*** jaosorior has joined #openstack-mistral06:57
d0ugalrakhmerov: hey, I didn't really have time to look at it further sorry07:05
rakhmerovd0ugal: np07:05
rakhmerovd0ugal: what's therve's time zone?07:06
rakhmerovI wonder when he usually appears online07:06
d0ugalrakhmerov: I think he is in Paris07:07
rakhmerovooh, ok07:07
d0ugalbut I don't know what hours he normally works :)07:07
*** shardy has joined #openstack-mistral07:07
d0ugalbut I guess he will be around soon.07:07
rakhmerovok, I'll wait07:07
rakhmerovnp07:07
d0ugalrakhmerov: oh, don't wait :)07:08
d0ugalrakhmerov: I just checked the vacation calendar, he is away this week.07:08
rakhmerovaaah07:08
rakhmerovok, np07:08
rakhmerovthen I'll take over this patch07:08
d0ugalcool07:08
rakhmerovwe need to fix it asap07:08
d0ugalrakhmerov: 1.1.8 might help?07:09
rakhmerovis it already issued?07:10
d0ugalyup07:10
rakhmerovwe need to try07:10
d0ugalI'll try now.07:10
rakhmerovdid you see the release notes? Is there anything about our issue maybe?07:10
rakhmerovok07:10
d0ugalhttp://docs.sqlalchemy.org/en/latest/changelog/changelog_11.html#change-1.1.807:10
d0ugalLooks related.07:11
rakhmerovd0ugal: because again, what therve has fixed in the patch seems to be a bug in SQLA07:11
rakhmerovd0ugal: yeah, that is our issue I think07:12
rakhmerovis 1.1.8 already in global requirements?07:12
rakhmerovok, it's SQLAlchemy>=1.0.10 so it should be pulled07:13
*** sharatss has joined #openstack-mistral07:14
rakhmerovand in upper-constraints.txt it's SQLAlchemy===1.1.807:14
* d0ugal is having internet issues07:16
rakhmerovd0ugal: I just rechecked on of the failing patches07:16
d0ugalcool, fingers crossed :)07:16
rakhmerovas far as I understand it should pull the new SQLA07:17
d0ugalyup, I think so07:17
d0ugalmy tests locally did - waiting for them to finish07:17
rakhmerov:)07:17
*** amoralej|off is now known as amoralej07:20
d0ugalrakhmerov: I am still seeing some failures07:22
d0ugalbut only a couple07:22
rakhmerovd0ugal: hey, the result is the same as after therve's patch07:22
rakhmerovwhich is good07:22
d0ugalhttp://paste.openstack.org/show/605203/07:22
rakhmerovthat's a different thing07:24
rakhmerovhttp://logs.openstack.org/96/452396/1/check/gate-mistral-python35/8a56885/testr_results.html.gz07:24
d0ugaland the same two failed for me in python 207:25
d0ugalso why are more failing in CI? :)07:25
rakhmerovin CI there's the same issue everywhere related to charset07:26
rakhmerovin API controller tests07:26
rakhmerovI like this issue much better07:26
rakhmerov:)07:26
d0ugalhah07:27
rakhmerovd0ugal: my assumption is that your local failures is a result of bad test design, I've seen similar (or same) issues with these expiration policy tests07:29
rakhmerovI believe we even have a bug to fix this test07:30
d0ugalI see07:30
rakhmerovto eliminate race conditions07:30
d0ugalI've not seen them fail locall before (only the tz bug that thrash|g0ne fixed)07:30
d0ugallocally*07:30
rakhmerovtry to run them several times07:30
d0ugalI do :)07:30
d0ugalbut I can try again, if they are unstable we should fix that.07:31
rakhmerovyes07:32
rakhmerovd0ugal: I'll take a look at this charset issue..07:32
rakhmerovit should be something simple07:32
openstackgerritKupai József proposed openstack/mistral master: Limit the number of finished executions.  https://review.openstack.org/44668007:35
openstackgerritRenat Akhmerov proposed openstack/mistral master: Explicitly set charset to UTF-8 in rest_utils for webob.Response  https://review.openstack.org/45262007:40
rakhmerovd0ugal: this should help07:41
rakhmerovI guess something has recently changed in webob and it now always requires UTF-807:41
rakhmerovsorry, not utf-807:41
rakhmerova charset rather..07:41
d0ugalaha, cool07:48
rakhmerovyes, it did help07:48
rakhmerovI'd suggest we merge it asap07:48
rakhmerovmgershen: hi, can you please look at https://review.openstack.org/#/c/452620/? We need to merge it asap to unblock development07:48
mgershensure07:51
rakhmerovthx07:51
mgershenyou want workflow+1 even though there are no tests yet?07:52
mgershenrakhmerov ^07:52
rakhmerovyes, no need in any special tests I think. Before this patch a number of tests were failing due to some requirements update07:53
rakhmerovthis patch fixed them07:53
rakhmerovthanks07:54
rakhmerovonce it's merge we'll be able to continue merging patches07:54
mgershenOK. I meant waiting for tests to run (not writing new tests), but just as well...07:54
rakhmerovmgershen: when I told you about this patch I already saw that unit tests passed :)07:56
rakhmerovI believe functional tests will be ok too07:56
mgershenok, let me know if you need me to review other things.07:57
*** jpich has joined #openstack-mistral08:00
*** openstackgerrit has quit IRC08:03
*** shardy has quit IRC08:09
*** tuan__ has joined #openstack-mistral08:19
tuan__Morning folks,08:19
d0ugalHey08:19
tuan__based on the discussion of expired tokne08:20
tuan__what about this bp08:20
tuan__https://specs.openstack.org/openstack/keystone-specs/specs/keystone/ocata/allow-expired.html08:20
tuan__i saw that keystone allow the expired token and this patched in actually merged in Octaka08:20
tuan__i am asking this question in keystone channel and hope they can have the right answer for us08:21
*** jtomasek has joined #openstack-mistral08:22
*** shardy has joined #openstack-mistral08:31
d0ugaltuan__: this is why an email thread would be good - then we can have input from different projects.08:32
tuan__d0ugal: yep08:34
tuan__and i am still waiting for it08:34
tuan__haha, i will update more information when i have answer08:34
*** bobh has joined #openstack-mistral08:38
*** bobh has quit IRC08:43
*** shardy has quit IRC08:46
*** shardy has joined #openstack-mistral08:48
*** szaher has quit IRC09:25
*** szaher has joined #openstack-mistral09:30
*** szaher has quit IRC09:33
*** szaher has joined #openstack-mistral09:39
*** openstackgerrit has joined #openstack-mistral09:39
openstackgerritMerged openstack/mistral master: Explicitly set charset to UTF-8 in rest_utils for webob.Response  https://review.openstack.org/45262009:39
openstackgerritAnastasia Kuznetsova proposed openstack/mistral master: Fix work of task() without task name within on-clause cases  https://review.openstack.org/45134009:39
openstackgerritRenat Akhmerov proposed openstack/mistral master: Limit the number of finished executions.  https://review.openstack.org/44668009:40
openstackgerritRenat Akhmerov proposed openstack/mistral master: Remove unnecessary tearDown function in testcase  https://review.openstack.org/45239609:41
openstackgerritRenat Akhmerov proposed openstack/mistral master: Add check for idempotent id in tempest tests  https://review.openstack.org/45242009:42
openstackgerritRenat Akhmerov proposed openstack/mistral master: Remove unnecessary setUp function in testcase  https://review.openstack.org/45176009:42
openstackgerritRenat Akhmerov proposed openstack/mistral master: Dynamic workflow name evaluation.  https://review.openstack.org/43410609:42
openstackgerritRenat Akhmerov proposed openstack/mistral master: Role based resource access control - get executions  https://review.openstack.org/45012109:42
openstackgerritRenat Akhmerov proposed openstack/mistral master: Updated from global requirements  https://review.openstack.org/45140209:43
*** szaher has quit IRC10:01
*** szaher has joined #openstack-mistral10:04
*** szaher has quit IRC10:10
*** szaher has joined #openstack-mistral10:11
*** sharat has joined #openstack-mistral10:21
*** sharatss has quit IRC10:21
*** jkilpatr has joined #openstack-mistral11:01
*** thrash|g0ne is now known as thrash11:02
d0ugaltoure: ping for when you are around11:10
*** tuan__ has quit IRC11:11
*** jenner_ has joined #openstack-mistral11:13
jenner_heya11:14
d0ugaljenner_: hi!11:14
jenner_guys, is there a way to run mistral with a horizon dashboard without keystone?11:14
d0ugalGood question. I guess that requires running Horizon without Keystone - is that possible?11:15
jenner_I'm trying to figure out if we can use it as a general purpose WF engine, and visualising a couple of thing would be nice (even if not a hard requirement)11:16
d0ugalIt can be used as a general purpose WF engine - people do that.11:16
d0ugalbut I'm not sure if they use the mistral-dashboard.11:16
d0ugalrakhmerov: do you know?11:16
*** 7IZAAYSS1 has joined #openstack-mistral11:16
rakhmerovhm...11:16
jenner_oh, apparently there's a way: https://bugs.launchpad.net/horizon/+bug/1580116 (my google fu is weak today)11:17
openstackLaunchpad bug 1580116 in OpenStack Dashboard (Horizon) "Make sure Horizon can run without Neutron, Glance, Nova and Keystone" [Wishlist,Fix released] - Assigned to Steve McLellan (sjmc7)11:17
rakhmerovjenner_: yes, you can use it as a general purpose WF (our main use case by now, btw) but I believe Dashboard can't be used without Keystone for now11:18
jenner_rakhmerov: thanks11:18
rakhmerovjenner_: actually, we're developing a separate much more advanced UI tool for Mistral now11:19
rakhmerovwhich is going to be a standalone tool (not part of Horizon)11:19
rakhmerovit's not open sourced yet though11:19
d0ugaland there is no public information either :)11:19
* d0ugal wants to see it11:19
rakhmerovyeah, just not yet11:19
rakhmerovI saw internal demos, and it's supercool already11:20
rakhmerovbut there's no public info yet, right11:20
rakhmerovhopefully soon (1-2 months)11:20
jenner_nice11:23
jenner_rakhmerov: are you going to anounce it somewhere sometime (if at all)?11:23
rakhmerovyes we are11:24
rakhmerovI can't tell now where and how though11:25
openstackgerritDougal Matthews proposed openstack/mistral-specs master: [WIP] Workflow Error Analysis  https://review.openstack.org/44321711:29
rakhmerovjenner_: getting back to your question about Keystone. Maybe there's a way but I've never done it. Mistral itself can work with auth disabled (or using Keycloak), I'm just not sure how it's going to work with Horizon, e.g. we still need info about tenants etc.11:29
rakhmerovso seems like it's mostly a question to Horizon if it supports this case11:30
jenner_rakhmerov: I see, thanks... looks like I'll just wait a bit for an alternative or skip the UI part as we don't really need it urgently11:31
rakhmerovok11:31
rakhmerovjenner_: feel free to ask any questions here or in the mailing list11:31
jenner_I definitely will, I actually have a list of questions for our decision matrix, but I'd rather don't bother you guys with it :)11:33
*** bobh has joined #openstack-mistral11:37
*** shardy is now known as shardy_lunch11:38
d0ugalmgershen: hey, if you have time could you take a look at https://review.openstack.org/446996 ?11:39
openstackgerritMerged openstack/mistral master: Fix work of task() without task name within on-clause cases  https://review.openstack.org/45134011:41
jenner_I've might hit an ubuntu bug here, but it seems like there are not std actions defined after I install mistral-(api|engine|executor) :(11:46
jenner_not even std.echo11:46
rakhmerovjenner_: may be you need to run mistral-db-manage populate11:48
jenner_rakhmerov: it errors out 13:29:01 openstackgerrit | Dougal Matthews proposed openstack/mistral-specs master: [WIP] Workflow Error Analysis   │ dmellado11:54
jenner_ups, sorry11:54
dmelladojenner_: wasup?11:54
jenner_rakhmerov: it errors out - https://dpaste.de/H7mR11:55
jenner_dmellado: wrong paste buffer, sorry :)11:55
dmelladonp! ;)11:55
*** dprince has joined #openstack-mistral12:00
jenner_rakhmerov: yeah, that's a confirmed bug, looks like I'm better off installing mistral from sources anyway12:01
jenner_the latest master seems to be working (pip install), however "auth_enable = False" in [pecan] section seems to be ignored -- I keep getting "ERROR (app) Operation not allowed"12:26
mgershend0ugal: approved. Looks great!12:27
*** bobh has quit IRC12:27
openstackgerritMerged openstack/mistral-specs master: Mistral Actions Library  https://review.openstack.org/44699612:32
*** jaosorior is now known as jaosorior_brb12:36
d0ugalmgershen: thanks!12:40
d0ugaljenner_: interesting, that should work - I tried it fairly recently.12:42
d0ugalbut not for a week or so12:42
*** shardy_lunch is now known as shardy12:45
*** fultonj has joined #openstack-mistral12:47
jenner_https://github.com/openstack/mistral/blob/master/mistral/api/access_control.py#L64 checks for auth_type but not for cfg.CONF.pecan.auth_enable like in #L2812:51
jenner_changed #L64 to "if not cfg.CONF.pecan.auth_enable or cfg.CONF.auth_type != 'keystone':" and it now seems to work, dunno if it breaks something else though :(12:53
*** amoralej is now known as amoralej|lunch12:54
openstackgerritDougal Matthews proposed openstack/mistral master: Don't create actions with empty arg_lists  https://review.openstack.org/41243312:59
jenner_btw, any thoughts about defining the API via OAI (swagger 2) or RAML?13:01
openstackgerritDougal Matthews proposed openstack/mistral master: Don't create actions when inspection fails  https://review.openstack.org/41243313:02
d0ugaljenner_: can you open a bug about the auth_type?13:03
d0ugaljenner_: It would be need to look at oai/raml. I have heard of the projects but otherwise don't have much experience with them.13:04
jenner_d0ugal: we're currently specifying/documenting all APIs with RAML, but it looks like OAI is the way to go, since it's now backed by a couple of big players - you basically define an API in YAML or JSON which allows you to 1) generate docs 2) create automatic tests 3) validate and convert URI and query parameters etc13:08
*** rbrady has joined #openstack-mistral13:14
*** rbrady has quit IRC13:14
*** rbrady has joined #openstack-mistral13:14
*** rook|tower is now known as rook13:19
jenner_d0ugal: https://bugs.launchpad.net/mistral/+bug/167918013:25
openstackLaunchpad bug 1679180 in Mistral "pecan.auth_enable ignored when running without keystone" [Undecided,New]13:25
jenner_duh, I thought launchpad supported markdown :(13:27
openstackgerritDougal Matthews proposed openstack/mistral master: Add documentation for the engine commands  https://review.openstack.org/43543813:28
d0ugaljenner_: nen, no worries - it is easy to read. Thanks!13:29
d0ugalLauchpad doesn't support fancy modern things like markdown ;)13:29
*** catintheroof has joined #openstack-mistral13:31
*** jaosorior_brb is now known as jaosorior13:33
tourerbrady chrome just died13:36
openstackgerritMerged openstack/mistral master: Updated from global requirements  https://review.openstack.org/45140213:52
jenner_so guys, what happens when I have a long-running WF execution and one of the actions used in it but not called yet is updated in between?13:57
jenner_also, is there some sort of crash recovery?13:57
*** jkilpatr has quit IRC14:02
*** bobh has joined #openstack-mistral14:03
toured0ugal pong14:12
d0ugaltoure: hey, it was just to let you know that I updated your spec14:15
d0ugaltoure: I was curious to find out why it failed CI :)14:16
toured0ugal which one of the specs?14:19
toured0ugal nm14:19
*** amoralej|lunch is now known as amoralej14:20
d0ugaltoure: do you have more than one? :)14:20
toureyeah but there is only one for mistral14:24
toure:)14:24
tourethanks for reviewing it14:24
toured0ugal thanks the patch :)14:25
d0ugalnp14:27
tourenothing like steering at something so long that you don't see the small problems14:28
toure:)14:28
d0ugaltoure: I had to compre it with another spec to find the missing .. at the start14:29
*** bobh has quit IRC14:30
toureyeah, I would have never found that :)14:30
toureI think the template maybe broken then, as that is what I started from14:30
* toure checks templates14:31
tourenope just my spec was missing that, I guess I figured that was just odd characters on top of the doc :14:32
toure:p14:32
openstackgerritAnastasia Kuznetsova proposed openstack/mistral master: Change Http action result content encoding  https://review.openstack.org/44914314:36
*** chlong has joined #openstack-mistral14:45
*** histrio has quit IRC14:49
*** sharat is now known as sharatss14:50
rakhmerovjenner_: what do you mean by "action updated in between"?14:52
rakhmerovjenner_: as far as crash recovery, what kind of crash do you mean? Crash of mistral components or failing workflow actions?14:54
*** histrio has joined #openstack-mistral14:55
rakhmerovas far as crashing mistral components, it's not 100% able to recover in certain cases14:55
rakhmerovwe're improving it now14:56
*** bobh has joined #openstack-mistral14:56
jenner_rakhmerov: imagine a WF that runs two days, at the end of day one a task is updated (code-wise) that has not been executed yet, does run the updated task or the one it saw right after it was executed?14:56
jenner_rakhmerov: IOW workflow versioning - yes or no? :)14:56
jenner_s/does run/does mistral run/14:57
rakhmerovWF is not going to change14:57
rakhmerovbut actions can change :)14:57
rakhmerovif you redeploy actions in the middle14:58
jenner_rakhmerov: ok, so if I update the WF definition itself, then it will still run the old one until it reaches the end state?14:58
jenner_ok14:58
rakhmerovtrue14:58
rakhmerovyes14:58
rakhmerovbut actions are associated with python code14:58
rakhmerovand it can be redeployed and then it'll run the new code14:59
jenner_crash recovery: one of the executors dies (e.g. the whole machine goes down for some reason) while running a WF15:00
rakhmerovok15:00
rakhmerovin this case the worst thing that can happen: some action can be executed twice15:00
d0ugalMeeting time!15:00
rakhmerovif it's idempotent that's not a problem15:01
jenner_ok15:01
rakhmerovteam meeting at #openstack-meeting-315:01
jenner_have fun :)15:01
jenner_thanks for the infos15:01
d0ugalthrash, toure, apetrich - meeting ;)15:02
toureyup15:03
jenner_hm, but who's recovering the crashed execution? 1) can another executor handle it 2) the crashed executor can pick up where it left after e.g. reboot or redeployment?15:04
openstackgerritMerged openstack/mistral master: Role based resource access control - get executions  https://review.openstack.org/45012115:13
rakhmerovjenner_: another executor15:22
*** sharatss has quit IRC15:33
jenner_rakhmerov: so it's "at least once" for idempotent WFs, non-idempotent WFs might end up calling tasks multiple times, no rollback or other automatic recovery, right?15:36
*** 7IZAAYSS1 has quit IRC15:36
rakhmerovjenner_: you got it right, it's "at least once"15:45
rakhmerovnope, we have a big workstream that is about to start to address more tricky crash scenarios15:46
rakhmerovliterally in 1-2 weeks we'll be working more actively on that15:46
rakhmerovso in general, that workstream will be devoted to improving scalability and HA (and all that HA assumes including recovery from various bad situations)15:48
*** rbrady is now known as rbrady-afk15:48
rakhmerovjenner_: I'm off for today, I'll be available tomorrow if you need to talk more15:49
rakhmerovthanks for your interest15:49
rakhmerovyou can also write emails to openstack-dev (with tag [mistral]) or my personal email too (renat.akhmerov@gmail.com)15:50
*** jkilpatr has joined #openstack-mistral16:18
*** d0ugal has quit IRC16:19
*** jpich has quit IRC16:24
*** jkilpatr has quit IRC17:05
*** chlong has quit IRC17:05
*** d0ugal has joined #openstack-mistral17:15
*** dprince has quit IRC17:17
*** chlong has joined #openstack-mistral17:20
*** jaosorior is now known as jaosorior_away17:29
*** shardy is now known as shardy_afk17:47
*** jamielennox is now known as jamielennox|away17:50
*** amoralej is now known as amoralej|off17:57
*** d0ugal has quit IRC17:59
*** dprince has joined #openstack-mistral18:02
*** harlowja has joined #openstack-mistral18:12
*** d0ugal has joined #openstack-mistral18:21
openstackgerritToure Dunnon proposed openstack/python-mistralclient master: [WIP] Workflow Error Analysis Client updates.  https://review.openstack.org/45290119:07
*** d0ugal has quit IRC19:23
*** d0ugal has joined #openstack-mistral19:25
*** bobh has quit IRC19:29
*** thrash is now known as thrash|brb19:33
*** d0ugal has quit IRC19:36
*** chlong has quit IRC19:37
*** harlowja has quit IRC19:37
*** chlong has joined #openstack-mistral19:51
*** thrash|brb is now known as thrash19:56
*** toure is now known as toure|biab19:57
*** shardy_afk has quit IRC20:02
*** shardy_afk has joined #openstack-mistral20:04
*** jamielennox|away is now known as jamielennox20:05
*** jkilpatr has joined #openstack-mistral20:25
*** harlowja has joined #openstack-mistral20:53
*** rbrady-afk is now known as rbrady20:54
*** dprince has quit IRC21:00
*** d0ugal has joined #openstack-mistral21:01
*** dprince has joined #openstack-mistral21:01
*** shardy_afk has quit IRC21:08
*** dprince has quit IRC21:14
*** chlong has quit IRC22:38
*** catintheroof has quit IRC22:44
*** bobh has joined #openstack-mistral23:35
openstackgerritLingxian Kong proposed openstack/mistral master: Role based resource access control - delete executions  https://review.openstack.org/45125523:43
openstackgerritLingxian Kong proposed openstack/mistral master: Role based resource access control - update executions  https://review.openstack.org/45116023:43

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!