Wednesday, 2019-11-06

mordredmnasiadka, yoctozepto: we should also likely do something in sdk so that calls to create_endpoint get a system-scoped token for you for their calls so that you don't have to have a separate clouds.yaml entry00:02
mordred(the separate entry is a good way to ensure this all works for now - but that seems like a pretty terrible UI to ask you to do for this)00:02
openstackgerritMerged openstack/python-openstackclient master: common: autogenerate docs  https://review.opendev.org/69198900:33
openstackgerritMerged openstack/python-openstackclient master: openstack.cli: autogenerate docs  https://review.opendev.org/69291400:33
openstackgerritMerged openstack/python-openstackclient master: compute: autogenerate docs  https://review.opendev.org/69291600:34
*** slaweq__ has joined #openstack-sdks00:40
*** slaweq has joined #openstack-sdks00:42
*** slaweq__ has quit IRC00:45
*** ricolin has joined #openstack-sdks00:47
*** slaweq_ has joined #openstack-sdks00:49
*** slaweq has quit IRC00:50
*** ricolin has quit IRC00:56
*** slaweq_ has quit IRC01:06
*** slaweq_ has joined #openstack-sdks01:08
*** dayou has quit IRC01:18
*** slaweq__ has joined #openstack-sdks01:20
*** slaweq__ has quit IRC01:20
*** slaweq has joined #openstack-sdks01:20
*** slaweq_ has quit IRC01:20
*** slaweq has quit IRC01:20
*** slaweq has joined #openstack-sdks01:21
*** slaweq has quit IRC01:22
*** slaweq has joined #openstack-sdks01:22
openstackgerritBence Romsics proposed openstack/openstacksdk master: Add router add/remove route operations  https://review.opendev.org/67432401:27
openstackgerritBence Romsics proposed openstack/openstacksdk master: Handle HTTP errors in add/remove router interface calls  https://review.opendev.org/68730401:27
*** ricolin has joined #openstack-sdks01:30
*** openstackstatus has joined #openstack-sdks01:44
*** ChanServ sets mode: +v openstackstatus01:44
*** dayou has joined #openstack-sdks02:14
*** ricolin has quit IRC02:19
*** ricolin has joined #openstack-sdks02:52
*** ricolin has quit IRC03:08
openstackgerritBence Romsics proposed openstack/python-openstackclient master: Add command: router add/remove route --route  https://review.opendev.org/67432503:30
*** ricolin has joined #openstack-sdks03:37
openstackgerritMerged openstack/python-openstackclient master: Add redirect testing  https://review.opendev.org/69292903:55
openstackgerritMerged openstack/python-openstackclient master: Deflate .htaccess  https://review.opendev.org/69293003:55
*** slaweq has quit IRC04:19
*** ricolin_ has joined #openstack-sdks04:22
*** ricolin has quit IRC04:25
*** ricolin_ has quit IRC04:35
*** goldyfruit_ has quit IRC04:47
*** slaweq has joined #openstack-sdks05:06
*** irclogbot_3 has quit IRC05:34
*** irclogbot_1 has joined #openstack-sdks05:38
openstackgerritMerged openstack/python-openstackclient master: Refactor AggregateTests  https://review.opendev.org/69307305:59
mnasiadkamordred: agreed, now it doesn't really work at all - when I do not supply domain in clouds.yaml - keystone complains it needs one, if I do - the token is not system scoped. So I guess in both cases it doesn't really do a system scoped token... I'll raise a bug.06:24
*** ricolin_ has joined #openstack-sdks07:02
*** jawad_axd has joined #openstack-sdks07:17
*** slaweq_ has joined #openstack-sdks07:34
*** slaweq has quit IRC07:36
*** yolanda has quit IRC07:55
*** gtema has joined #openstack-sdks07:59
*** tosky has joined #openstack-sdks08:10
*** yolanda has joined #openstack-sdks08:11
*** ricolin_ is now known as ricolin08:28
*** ralonsoh has joined #openstack-sdks08:46
*** jpena|off is now known as jpena08:48
*** slaweq_ has quit IRC08:55
*** ricolin has quit IRC08:57
*** jpich has joined #openstack-sdks09:12
*** gtema has quit IRC09:13
*** gtema has joined #openstack-sdks09:49
*** gtema has quit IRC09:54
*** cdent has joined #openstack-sdks09:58
*** slaweq_ has joined #openstack-sdks10:00
*** slaweq__ has joined #openstack-sdks10:04
*** slaweq_ has quit IRC10:05
*** slaweq__ has quit IRC10:10
*** gtema has joined #openstack-sdks10:17
fricklerpython-novaclient 16.0 seems to be breaking heat and therefore sdk testing https://zuul.opendev.org/t/openstack/build/2c83d0168fda4384837b676d336694ed/log/controller/logs/screen-h-eng.txt.gz#1379-141011:37
fricklerhttps://zuul.opendev.org/t/openstack/builds?job_name=openstacksdk-functional-devstack11:38
*** gtema_ has joined #openstack-sdks12:26
*** jpena is now known as jpena|lunch12:27
*** jpena|lunch is now known as jpena13:21
efriedfrickler: anything I can do about that novaclient breakage?13:28
efried(I haven't looked at it yet, but I was following the steps that led up to that requirements patch being mergeable)13:29
*** noama has joined #openstack-sdks13:35
*** slaweq__ has joined #openstack-sdks13:39
*** slaweq has joined #openstack-sdks13:43
*** slaweq__ has quit IRC13:44
*** gtema_ has quit IRC14:04
*** goldyfruit_ has joined #openstack-sdks14:10
*** slaweq has quit IRC14:31
*** goldyfruit___ has joined #openstack-sdks14:38
*** goldyfruit_ has quit IRC14:40
openstackgerritPedro Henrique Pereira Martins proposed openstack/keystoneauth master: Fixes OIDC authentication with multiple IdPs  https://review.opendev.org/69214014:43
*** dave-mccowan has joined #openstack-sdks14:49
*** slaweq has joined #openstack-sdks14:51
*** dave-mccowan has quit IRC14:54
*** slaweq has quit IRC14:55
*** mriedem has joined #openstack-sdks15:00
fricklerefried: there's a fix proposed in heat now https://review.opendev.org/693165 but I see you already reviewed it15:01
efriedyup15:01
efriedif a quicker fix were needed, I guess I would suggest capping the novaclient requirement there15:02
efriedbut that fix seems straightforward enough.15:02
fricklerefried: yes, I think I'd be fine waiting a couple of days for heat to get this merged, but it would be good to add some test/job into reqs to prevent this from happening again15:04
efriedgetting novaclient to gate on all possible consumers though...15:05
mriedemthe reqs job failed on horizon and i fixed osc since i noticed it was going to break but assumed the reqs job tested heat15:05
efriedoh, you're talking about getting reqs to gate on heat15:05
mriedemyeah15:05
efriedthat's probably not a bad plan.15:05
mriedemhttps://review.opendev.org/#/c/690097/15:05
mriedemit's definitely a slippery slope, because watcher, designate, senlin, blazar, etc15:06
efriedwould heat's CI have caught this though? I thought we first noticed when sdk was running heat running novaclient...15:06
efriedyeah, that15:06
fricklerefried: heat func tests are broken now, too15:06
efriedokay.15:06
mriedemheat's ci is gating on upper-constraints using novaclient15:07
mriedemand the reqs for u-c just recently merged15:07
efriedI mean, if it wasn't, that would point to a hole in heat's CI15:07
fricklersee e.g. https://review.opendev.org/#/c/693134/15:07
efriedokay, so is someone going to propose a reqs zuul.yaml change to make it run some heat?15:07
fricklerheat might install some -tips job for their deps15:07
fricklerso they notice upcoming breakage earlier15:08
efriedalso not a terrible plan15:08
fricklerthe other option would be to drop heat tests from the sdk jobs and transitively from devstack15:08
efriedwouldn't that just lead to field escapes?15:09
frickleror move them into a seperate non-voting jobs at least15:09
fricklerwhat does "field escapes" mean in this context?15:09
efried"we don't notice the problem until a customer hits it"15:09
efriedi.e. "bug escapes to the field"15:10
efriedmaybe I'm misunderstanding, but the point isn't for us to be able to continue to merge heat code despite a broken dep. The point is to avoid the broken dep, as early as possible.15:10
fricklerfor me the point would be to be able to merge sdk and devstack patches even while heat isn't keeping up with things. though I admit I'm slightly biased on this because it's also heat blocking my pet "drop identity admin endpoint" project15:12
mriedemha15:12
mriedemthe truth comes out15:13
fricklerbut o.k., I can propose to add some heat job, or maybe even better the same sdk job that is also running on devstack, to reqs, then see what comes out of that15:15
*** gtema has quit IRC15:17
fricklerif you want to watch or comment: https://review.opendev.org/693197 Add openstacksdk-functional-devstack job15:22
*** gtema has joined #openstack-sdks15:32
*** slaweq has joined #openstack-sdks16:01
*** slaweq has quit IRC16:07
*** cdent has quit IRC17:01
*** gtema has quit IRC17:01
*** efried is now known as efried_rollin17:11
*** jawad_axd has quit IRC17:23
*** jawad_axd has joined #openstack-sdks17:29
openstackgerritPedro Henrique Pereira Martins proposed openstack/keystoneauth master: Fixes OIDC authentication with multiple IdPs  https://review.opendev.org/69214017:29
*** melwitt is now known as jgwentworth17:31
*** jawad_axd has quit IRC17:32
*** jpich has quit IRC17:34
*** dtantsur is now known as dtantsur|afk17:38
*** jpena is now known as jpena|off18:06
*** mriedem is now known as mriedem_lunch18:15
umbSublimeHi again, would this be the right place to discuss about ospurge ?18:23
*** jgwentworth is now known as melwitt18:36
*** tosky has quit IRC18:48
*** mriedem_lunch is now known as mriedem18:57
openstackgerritPedro Henrique Pereira Martins proposed openstack/python-openstackclient master: Add documentation about login with federation  https://review.opendev.org/69323219:10
openstackgerritRodolfo Alonso Hernandez proposed openstack/openstacksdk master: "qos_network_policy_id" attribute added to port resource  https://review.opendev.org/69324720:06
*** gtema has joined #openstack-sdks20:15
*** ralonsoh has quit IRC20:19
*** gtema has quit IRC20:19
*** efried_rollin is now known as efried20:24
efriedumbSublime: Shanghai would probably be the right place this week20:26
umbSublimeefried: I wish I was there20:27
efriedthat makes one of us20:27
umbSublimeLast summit I was too was the one in Boston20:27
efriedthat might have been my first one20:28
efriedno, that can't be right20:28
* umbSublime 's first and only summit20:29
efriedAustin or Boston20:29
*** goldyfruit___ has quit IRC21:16
*** efried is now known as efried_afk21:19
*** efried_afk is now known as efried21:41
openstackgerritMichael Johnson proposed openstack/python-openstackclient master: Switch to using osc_lib.utils.tags  https://review.opendev.org/66286421:50
johnsomWould really be good to merge ^^^^ as changes are being made to the old _tag.py that this removes.21:51
johnsomefried So, I see your change to _tag merged that added the "enhance_help parameter to some of the tags methods. I am guessing those need to be added to the new osc_lib version?21:55
johnsomhttps://www.irccloud.com/pastebin/Gj2aSRX5/21:56
efriedjohnsom: ugh, that's an ugly enough hack kept buried in python-openstackclient, I would hate for it to have to bleed over to a different repo21:56
johnsomefried I had removed _tag.py from pythonclient, it has just been sitting for review for a while.21:56
efriedI didn't know about that moving to osc-lib21:56
johnsomWe moved all of that code to osc_lib/utils/tags.py21:57
johnsomSo that other projects could use it21:57
efriedOut of curiosity, what other projects would need to use it?21:57
efriedSince it's you talking, I'm guessing octavia :P21:57
johnsomOctavia wants it for example21:57
johnsomIt would be nice if we all implemented --tags the same... grin21:58
johnsomFYI: https://review.opendev.org/#/c/662859/21:59
johnsomI'm not sure I quite understand why the lambda is needed, but I just found that the removal patch was in conflict22:00
johnsomThis is the patch I just rebased that will likely fail as osc_lib doesn't have the "enhance_help" param: https://review.opendev.org/#/c/662864/22:05
efriedjohnsom: I can at least explain how this came about.22:08
efriedThe networking commands are kind of a mess. The commands and options for neutron and nova-network are interleaved.22:09
johnsomOk. I am guessing it is part of getting the docs working22:09
johnsomJoy22:10
efriedWhen you're running on the CLI, the client tries to detect which network you're running by looking for the neutron service.22:10
efriedand if it doesn't find it, assumes you're running nova-net22:10
efriedthen it loads up only the commands/options for whichever service it decided you're running.22:10
*** tosky has joined #openstack-sdks22:10
efriedthis of course relies on you being authorized to ask the right question to look for the service. But I guess chances are if you can't look at the catalog, you can't do anything else on the CLI, so it may not matter22:11
efriedanyway, that didn't affect the docs when they were hardcoded...22:11
johnsomWell, if I remember right, you could run both at the same time22:11
efriedoh, well, osc wouldn't let you22:11
efriedbecause if it detected neutron, it would only load up the neutron options and operations.22:12
efriedI'm asserting this based on code inspection, not experience btw22:12
efriedso my goal was to get the docs generated22:13
efriedBUT22:13
efriedif you're looking at the HTML docs, on a web server somewhere, we have NO way of knowing whether you care about neutron or nova-net.22:13
efriedso we have to show docs for both22:13
johnsomSo nova networking is EOL I think, maybe we just have the docs render the neutron version?22:13
efriedexcept osc has to continue to work against old clouds. Forever.22:14
efried...so I had to come up with a way to make cliff generate both sets of docs22:14
efriedand this was before I got to the keystone and glance docs, where I actually ended up completely separating them into different pages.22:14
johnsomIt would still *work*...  wink, wink22:14
efried...for e.g. identity v2 vs v3, glance v1 vs v222:15
efriedbut that was a much easier sell in that case because the different versions were handled by completely different code, separated cleanly by package.22:15
efriedin order to do that for the networking pieces, one would first have to split them into at least separate classes.22:16
efriedIn any case, what I ended up doing was implementing this three-way code path: We look for neutron, and if we don't find it because it's not there we assume nova-net, but if we don't find it because we can't even ask the question, we assume we're building docs.22:17
efriedIf neutron, we only run the methods that add neutron opts. If nova-net, likewise.22:19
efriedBut for the docs fork, we run both; and everywhere we register an option, we decorate its help with "neutron only" or "nova net only" kind of thing.22:19
efriedTo come full circle, this means that any code path that registers an option used by networking commands needs a way to understand *when* and *how* to add those decorations.22:20
efriedthe easiest way was with a callable kwarg22:20
efriedso that in fact only the caller has to know when/how.22:20
efriedAnd making the default a no-op lambda was easier than having22:21
efriedif enhance_help is None:22:21
efried   help=_(normal)22:21
efriedelse:22:21
efried   help=enhance_help(_(normal))22:21
efriedin each of the methods (tags wasn't the only one)22:21
johnsomSo, the odd thing is looking at the nova networks API, it doesn't even have tags22:21
johnsomhttps://docs.openstack.org/api-ref/compute/?expanded=create-allocate-floating-ip-address-detail#create-allocate-floating-ip-address22:21
johnsomRight, ok, so that is this use case.22:23
efriedplease also keep in mind that I was trying to make $new_docs look as much like $old_docs as possible with this series. I wasn't getting in the business of figuring out where $old_docs were incorrect or whatever.22:24
johnsomRight22:24
efried...except insofar as $old_docs had gotten left behind when new options were added, kind of thing.22:25
johnsomThere is a bunch of bit-rot in both the nova and neutron docs from what I have seen.. Fixing the API-ref for load balancing was a chore22:25
efriedwhich is one of the purposes of this transition: so you don't have to worry about maintaining two places.22:25
efriedIs dtroyer the/a maintainer of osc-lib?22:26
efriedProbably need to ask him whether he's willing to let this enhance_help kwarg cross the project boundary22:27
johnsomYeah I think so. He is the one I talked to when I did the tags move to osc_lib22:27
efriedSince the default is a no-op, it's probably not the worst thing in the world, shouldn't hurt callers who don't care about it.22:27
efriedI would want to see it documented.22:28
efried...but it looks like you put nice docstrings in the osc-lib incarnations, so yeah, just following that.22:28
johnsomYeah, I tried to make it better.22:29
efried++22:29
efriedI guess the way to go about this is to propose that modification to the osc-lib side and use that vehicle to get dtroyer's yea/nay.22:29
efriedYou want to do that since you did the original, or you want me to do it since I made this mess? (inadvertently, and sorry)22:30
johnsomWell, I didn't do a good job getting his attention to get the python-openstackclient patch merged, so....22:30
dtroyertbh I really don't like that approach but I don't have the time to do anything differrent.  the entire generated docs is a huge compromise anyway so wtf22:30
johnsomdtroyer Hi Dean22:30
dtroyerjohnsom: my apologies for that22:30
johnsomI am wondering for tags if we can make a parameter that is more generic and optional and move forward that way. That way for these edge cases we can pass something over to have an annotation.22:32
johnsomOk, well, at least I understand the issue now.22:32
johnsomefried Let me noodle on it a bit.22:32
efrieddtroyer: Yeah, the right thing is probably to split the neutron/nova-net commands/opts into distinct classes (preferably in distinct packages), but I'm not sure how the CLI would decide which entry points to load up.22:34
efrieduhh, maybe the framework is already there, because of how we use --os-*-api-versionZ22:34
dtroyerefried: the problem there is there is nobody to do that work.  That also needs to be done  for volume v2/v3 and has sat there for 2-3 years now22:36
dtroyerand I am now stuck trying to sort out how to deal with the SDK release breaking the snot out of nodepool and cleaning up the mess that made in my 3ci stacks22:37
johnsomI think we can make the tags code in osc_lib take a version qualifier and produce the same result, just by handling the tags params special on the OSC side. I think a generic parameter like "version_qualifier" would be ok for the shared tags code.22:39
efriedjohnsom: but then you have to duplicate the text, translation, formatting, and conditioning in osc-lib.22:41
efriedI suppose you could just *move* it in there, shrug.22:41
johnsomOne question I have is we don't call out api versions on any of the other commands right? We just assume the "not implemented" response is clear enough.22:43
efriedwhat do you mean?22:44
johnsomWell, here: https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/network.html#network-create22:44
johnsom--dns-domain only exists on certain versions of neutron API. We don't call that out here.22:45
johnsomSame with --qos-policy which is only available in certain versions of neutron API and with the neutron extension loaded.22:45
efriedYeah, that would be nice. Not a reason to move backwards on neutron vs nova-net though22:46
efriedand btw, the "not implemented" response is not necessarily clear.22:47
efriedas an osc noob I had a hell of a time figuring out that my under-the-covers microversion wasn't adequate, and how to remedy it.22:47
johnsomI'm just noticing that this is the first time we have added any version information to the python-openstackclient docs.22:48
efriedCan't imagine someone trying to figure out why they can't use the --subnet flag22:48
efriedoh, no it's not at all22:48
efriedThe previous version of these same docs had these same decorations.22:48
efriedstand by22:48
johnsomWeil, in server create, the micro-versions aren't called out for those commands22:49
efriedhttps://docs.openstack.org/python-openstackclient/train/cli/command-objects/network.html#network-create22:49
efriedno, microversions aren't. That would arguably be a nice enhancement. Ideally it would somehow tie back into the client code to discover that automatically, rather than being hardcoded.22:49
efriedI'm just saying for neutron vs nova-net, it *was* called out, which is why I didn't see it being a viable path forward to stop doing that when the docs were generated.22:50
efriedalso fwiw https://docs.openstack.org/python-openstackclient/train/cli/command-objects/role.html identity docs had v2 vs v3 markers.22:51
johnsomwere those static pages before?22:51
efriedthat's major version of course, not microversion.22:51
efriedyes22:51
johnsomah22:51
efriedoh, and the way I found out this was even a thing was by trying to build the docs by running both paths. That gives errors on conflicting opts. If you look at the rest of the offending patch, you'll see where I had to put in some pretty weird "if docs build" conditions in some places.22:52
johnsomYeah, I saw those22:53
efriedanyway, as dtroyer implied, there's a right way and there's an expedient way. Unless you're signing up to do the former (I'm not) we're pretty much stuck doing the latter, even though it's kind of distasteful.22:53
johnsomI think we can leave that nastiness in OSC. I'm just working on a way to allow passing that stuff into osc_lib in a more clean way, focused on the tags library for now.22:54
johnsomWell, no good dead goes unpunished. I volunteered to do the tags library work... lol22:54
efriedfor what it's worth, I played around with other "more clean" options when I was authoring the patch.22:55
efriedBelieve it or not, this turned out to be the cleanest, from a DRY standpoint.22:55
johnsomSo you are advocating for leaving duplicate code in OSC and osc_lib?22:56
efriedoh, sorry, no; I'm advocating adding the enhance_help kwarg in osc-lib, properly documented.22:56
efriedYeah, I guess there's two "expedient" solutions, but I was assuming "leave it like it is" (and abandon the removal patch) wasn't on the table.22:57
johnsomUgh, I really hate having to modify osc_lib again and start this cycle over and I'm not a fan of pulling forward this setup.23:14
openstackgerritEric Fried proposed openstack/osc-lib master: WIP: Add enhance_help kwarg to tags option generators  https://review.opendev.org/69326723:14
efriedjohnsom: well, fwiw there's the code side ^23:15
efriedI'll let you sleep on it :)23:16
efriedI'm out o/23:16
openstackgerritMichael Johnson proposed openstack/python-openstackclient master: Switch to using osc_lib.utils.tags  https://review.opendev.org/66286423:24
johnsomNot sure those jobs are going to pick up that tags library hack patch, so mine is probably going to fail out even with the depends-on.23:25
*** mriedem has quit IRC23:29
*** noama has quit IRC23:35
*** tosky has quit IRC23:48

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!