| *** harlowja has quit IRC | 00:00 | |
| *** malini1 has joined #openstack-qa | 00:11 | |
| devananda | mtreinish: hi! long reply to your comment on 100989 (move ironic api tests to admin/) just posted. | 00:15 |
|---|---|---|
| devananda | mtreinish: fwiw, i'm inclined to just copy-paste that to the ML as it seems like this will be useful for a wider audience | 00:16 |
| openstackgerrit | A change was merged to openstack/tempest: Added swift CLI test cases https://review.openstack.org/96386 | 00:19 |
| mtreinish | devananda: so I think the disconnect here is the assumption that your api will only ever be used by other openstack services | 00:22 |
| mtreinish | my comment was that if you have an api surface that you guarentee to be a stable interface you need really do a full path test using auth and everything | 00:22 |
| mtreinish | and it should work for any ironic deployment | 00:22 |
| mtreinish | it's fair game in an api test to allocate resources from another service if you need them passed into the api call | 00:23 |
| boris-42 | mtreinish hey there | 00:24 |
| boris-42 | mtreinish could you pls review spec about auto conf generation https://review.openstack.org/#/c/94473/ ? | 00:25 |
| boris-42 | mtreinish sorry for big delay was quite busy around tons of other stuff | 00:25 |
| mtreinish | boris-42: yeah, I'll take a look tomorrow (it's getting kind of late here) | 00:25 |
| mtreinish | sure no worries | 00:25 |
| boris-42 | mtreinish hm ) here is 4:25 am lol=) | 00:26 |
| mtreinish | boris-42: we just have different schedules I guess :) | 00:26 |
| boris-42 | mtreinish 24/7 is my=) | 00:26 |
| boris-42 | mtreinish btw I finished work on osprofiler | 00:27 |
| boris-42 | mtreinish https://github.com/stackforge/osprofiler it has finally good API | 00:27 |
| boris-42 | mtreinish and I released it today, so when it get in we can as well try to integrate it with tempest | 00:27 |
| mtreinish | boris-42: cool I'll take a look | 00:27 |
| boris-42 | mtreinish great | 00:27 |
| boris-42 | mtreinish seems like I should sleep a bit | 00:27 |
| boris-42 | mtreinish see you | 00:28 |
| devananda | mtreinish: "... used by other openstack services" for deployment | 00:31 |
| mtreinish | devananda: my concern is that just using a fake driver with a limited subset of the api isn't going to cover a wide enough area of the api server | 00:31 |
| *** Sukhdev has joined #openstack-qa | 00:31 | |
| devananda | mtreinish: we assume an operator will do CRUD operations on nodes | 00:31 |
| devananda | mtreinish: so those api tests are exercising what an operator will do | 00:31 |
| devananda | mtreinish: whereas the scenario tests exercise what another service (nova) will do | 00:32 |
| *** dmorita has joined #openstack-qa | 00:32 | |
| mtreinish | devananda: see the scenario tests are for testing functionality I agree, but what about the api that nova is using for example | 00:32 |
| mtreinish | where will that be tested | 00:32 |
| devananda | mtreinish: that is tested in the scenario test | 00:33 |
| devananda | well, exercised | 00:33 |
| mtreinish | yeah that's the difference | 00:33 |
| devananda | mtreinish: so the only thing nova does which the CRUD tests don't do is "set provision state" | 00:33 |
| mtreinish | that's why we have the split | 00:33 |
| devananda | which is illogical out side of the context of a scenario test, since it requires neutron, keustone, and glance (and, well, nova) | 00:33 |
| devananda | nova also performs a lot of R/U operations, which are tested in the api tests | 00:34 |
| *** jaypipes has quit IRC | 00:35 | |
| mtreinish | devananda: so ok, it's a small surface. My concern is that since you're using ironicclient (I'm assuming) in nova that you won't neccesarily catch api bugs with just the scenario tests | 00:35 |
| *** matsuhashi has joined #openstack-qa | 00:35 | |
| mtreinish | and it doesn't provide you as good protection against breaking the interface | 00:35 |
| devananda | the other confusing part is that creating a new resource in ironic - -within a running cloud -- will affect that cloud | 00:35 |
| devananda | the idea of running C..D operations against ironic's APi with a real driver (eg, pxe_ipmitool) and against a cloud that has users ... well ... i dont see how that could work today | 00:36 |
| devananda | because nova will likely see the new resources *too* and start scheduling work on them | 00:36 |
| devananda | even while tempest is trying to do API tests | 00:36 |
| mtreinish | devananda: there is no tenant support, or other binning of nodes? | 00:37 |
| devananda | in the future, IFF ironic were to add tenant<->node mapping (which we dont plan to) then such a test could be possible | 00:37 |
| devananda | correct | 00:37 |
| devananda | it's been loosely discussed, but i'm not aware of anyone with an interest in implementing that | 00:38 |
| *** Sukhdev has quit IRC | 00:38 | |
| devananda | it doesn't fit RS or RH or HP use cases | 00:38 |
| devananda | (that I'm aware of) | 00:38 |
| mtreinish | well you're the one who would know :) | 00:39 |
| mtreinish | so I guess it doesn't really make sense to run tests that directly hit the ironic api in a real deployment. But, I still think it's a useful thing for the gate | 00:39 |
| devananda | definitely | 00:40 |
| devananda | hence the fake driver | 00:40 |
| mtreinish | the only use case I see for a real deployment is validation when you first bring up ironic (before other things are running) | 00:40 |
| devananda | mtreinish: I can see that as well. Without any other services, the only hardware-specific things which one could curerntly test are "set power state" and "set console state" | 00:42 |
| devananda | the API for those can be tested with a fake driver just the same as with a real one -- the only difference is whether real hardware is going to complain if you input an incorrect password | 00:43 |
| *** coolsvap is now known as coolsvap|afk | 00:44 | |
| mtreinish | devananda: heh, are we going to be able to test that case in the api tests with the fake driver? | 00:45 |
| devananda | mtreinish: exception paths? we test some already with the fake driver: https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/fake.py | 00:46 |
| devananda | mtreinish: but i'm sure we could test more | 00:46 |
| devananda | mtreinish: woops, meant to link to line 93 | 00:47 |
| mtreinish | yeah but I meant the rest api surface when that gets raised | 00:49 |
| devananda | https://review.openstack.org/#/c/74065/11/tempest/api/baremetal/test_nodes_negative.py | 00:50 |
| devananda | mtreinish: so we need more tempest api tests to cover those exceptions -- some are proposed, eg ^ | 00:50 |
| devananda | mtreinish: but my point is, we can test them without real hardware | 00:50 |
| mtreinish | because the api tests have 2 goals ensuring api consitency in a real deployment and making sure the api did what it said it did (which we are a little sloppy/hand wavy about right now) | 00:50 |
| mtreinish | devananda: that's fine I just need to think about it a bit more because part of tempest's design philosphy is working on any deployment | 00:51 |
| devananda | mtreinish: how would it ensure api consistency in real deployments for admin-only apis? | 00:52 |
| devananda | mtreinish: hmm... so testing exception paths essentially binds the implementation of the fake driver to the API tests. I'm guessing that's not what you would want | 00:54 |
| devananda | *by depending on specific requests to trigger those exceptions | 00:54 |
| mtreinish | yeah that was my point before, that feels very unit test like | 00:55 |
| devananda | but the same would be true for any other driver | 00:55 |
| devananda | right ... it does | 00:55 |
| mtreinish | think of the refstack use case for example. Sure some of the response fields will be deployment specific but it can still check the others | 00:55 |
| devananda | yep. been trying to keep that in mind | 00:55 |
| devananda | but actually ALL of the fields will be deployment specific | 00:55 |
| mtreinish | heh, ok check response codes and that the keys look right in the json :) | 00:56 |
| *** morganfainberg is now known as morganfainberg_Z | 00:57 | |
| devananda | mtreinish: :) | 00:57 |
| mtreinish | how hard would it be to figure out the response given a large set of config information | 00:57 |
| mtreinish | is that feasible? | 00:57 |
| devananda | mtreinish: i guess another way to put my concern is that, if you introduce a dependency on real hardware (regardless of which driver is used) then the tests will have non-deterministic properties | 00:58 |
| devananda | becuase, well, hardware fails non-deterministically | 00:58 |
| mtreinish | devananda: so does everything in openstack... | 00:58 |
| devananda | lol :) | 00:58 |
| devananda | touche | 00:58 |
| mtreinish | all I think we need to do is make sure that we can run the tests on real hardware. Not that it's a good idea | 00:59 |
| devananda | mtreinish: given prior knowledge of all the hardware information, management credentials, etc, yes -- i could predict the output of a "node show $UUID" command | 00:59 |
| mtreinish | and we probably should document that because of the nature of what ironic is doing thinks will probably not work | 00:59 |
| mtreinish | devananda: would those be a reasonable thing to put in config (or some other loadable file) | 01:00 |
| devananda | mtreinish: "the keys to the kingdom" ? | 01:00 |
| mtreinish | and just have it default to the fake driver | 01:00 |
| mtreinish | yeah something like that, like have an option ironic-info.yaml config option. If it's present load it and use it for figuring out the api responses | 01:01 |
| mtreinish | if it's not just use the fake drivers values | 01:01 |
| devananda | mtreinish: so if it were possible to put, in some config file, all the actual hardware info, then run the API tests -- even if they dont actually deploy to that hardware | 01:01 |
| devananda | that would be acceptable? | 01:02 |
| mtreinish | what do you mean by don't actually deploy to that hardware. If ironic is running won't and you send it the api commands won't it do that? | 01:03 |
| devananda | as i said above, "set provision state" can't be successfully run without other services present (nova, glance, neutron, keustone) | 01:03 |
| mtreinish | ahh, ok yeah that would be fine then | 01:04 |
| devananda | most, if not all, other API end points could be exercised in isolation, i think | 01:04 |
| mtreinish | I guess that will always just be a gap in the api tests | 01:04 |
| devananda | not that they do anyuthing | 01:04 |
| devananda | to the hardware | 01:04 |
| devananda | i really don't understand the point still :( | 01:05 |
| mtreinish | I think I should take a look at the ironic code too, just to get a better feel for what it's doing. Because I'm just curious what would be required to directly test set provision state | 01:05 |
| devananda | mtreinish: take a look at what the nova.virt.ironic driver does prior to callign set provision state :) | 01:06 |
| devananda | mtreinish: there's interaction with neutron (plugging VIFs based on physical MACs retrieved from Ironic), passing a glance image ref to Ironic, and passing in the instane properties from the nova flavor | 01:06 |
| devananda | then during provisining, ironic depends on neutron to provide DHCP BOOT response pointing to the right tftp location | 01:07 |
| devananda | and downloading the images from glance, etc | 01:07 |
| mtreinish | yeah but I still think we could have a test, albeit a long one that does all of that. | 01:08 |
| devananda | heh, sure | 01:08 |
| adam_g | oh man | 01:08 |
| devananda | but then that test is very heavily dependent upon the nova and glance apis | 01:09 |
| * adam_g reads scrollback | 01:09 | |
| devananda | adam_g: hi! | 01:09 |
| devananda | yea, long scrolback... | 01:09 |
| mtreinish | yeah, and we test those in tempest too so they should be stable | 01:09 |
| mtreinish | and we can mark the test as requiring all those services | 01:09 |
| mtreinish | my concern with skipping the thing the api that does the cool stuff is it's basically tying all of ironic to be a nova driver | 01:09 |
| mtreinish | not that is necessarily a bad thing | 01:10 |
| adam_g | why would we be testing what the nova driver is doing internally? | 01:10 |
| devananda | slightly off topic, but i'll circle back -- if tempest is not able to verify the result of an action externally (eg, via soem other means than the service's API), is that still a valid test? | 01:11 |
| adam_g | RE: testing other drivers with loadable node details, as a next step: could we add a CONF.baremetal.enabled_driver option that defaults to 'fake', skip the API tests if that value is not in $supported_drivers and later add IPMI testing with node details loaded from an external file? | 01:11 |
| devananda | in looking at tempest/api/compute it looks like tempest ssh's into the instance to verify each change | 01:11 |
| adam_g | i think tripleo will be hitting this when they get to the point of running tempest against seed + undercloud. and they already have their kingdom keys in a json file, and i imagine they might be resist enabling the fake driver by default | 01:12 |
| mtreinish | adam_g: because it's a requirement for booting a node which is an exposed part of ironic's api, so at least ideally it should be tested | 01:12 |
| devananda | mtreinish: but most of ironic's api tests just ask ironic for the result and compare to the input | 01:12 |
| mtreinish | devananda: yeah it's valid test, not the best test but I don't think we'd block it | 01:12 |
| devananda | mtreinish: ack | 01:12 |
| devananda | mtreinish: all of our current api tests do that ^, fwiw | 01:13 |
| devananda | mtreinish: also to note, deploying an instance with ironic in the gate takes ~5 minutes | 01:13 |
| devananda | mtreinish: because nested virt PXE boot is sllooooowww | 01:13 |
| devananda | so "ssh into isntance to verify $thing" really can't be done in the gate very much for Ironic | 01:13 |
| devananda | on hardware it's actually *slower* in many cases | 01:14 |
| devananda | because there are two POST cycles per deploy | 01:14 |
| mtreinish | oh really 2, heh I could imagine just one being slower | 01:15 |
| devananda | s/because/and/ | 01:15 |
| clarkb | devananda: so slow | 01:15 |
| devananda | clarkb: hi! | 01:15 |
| clarkb | I ran our puppet in nested virt a couple weeks ago | 01:15 |
| clarkb | takes like 45 minutes | 01:15 |
| devananda | heh | 01:15 |
| adam_g | mtreinish, right, i guess its just a weird corner of the API thats really hard to test. having tempest emulate everything thats currently required of the driver kinda seems like we'd be making the test dependent on the stability of the drivers internals, or the other way | 01:16 |
| devananda | mtreinish: oh, and ^ would also tie the tempest test to a specific driver | 01:17 |
| mtreinish | wait the nova driver is dependent on the ironic driver too | 01:18 |
| devananda | s/tempest to/tempest config to/ | 01:19 |
| devananda | it's late and i'm typing too fast... sorry | 01:19 |
| adam_g | nova driver == ironic driver | 01:20 |
| devananda | nova.virt.ironic != ironic's node.driver | 01:20 |
| devananda | words are hard | 01:20 |
| mtreinish | no worries, I think it's a bit too late for me too | 01:20 |
| adam_g | doh, right | 01:20 |
| adam_g | haha | 01:20 |
| mtreinish | devananda: so honestly I'm fine with the current approach, I just don't want to say that will be it. *Long* term I'd like to find away to do all this magic | 01:21 |
| mtreinish | and I just don't want to box us into the corner too hard | 01:22 |
| mtreinish | just so we can get something gating quickly | 01:22 |
| adam_g | ill add a config flag to specify enabled drivers | 01:22 |
| adam_g | i really think we'll extend to real ipmi driver coverage with tripleo | 01:22 |
| mtreinish | adam_g: ok cool | 01:22 |
| devananda | adam_g: that'll be interesting | 01:23 |
| devananda | adam_g: power control only? or a full deploy w/o nova? | 01:23 |
| adam_g | devananda, just the basic API stuff we have now. i know the goal is to have a tempest run against each stage of the ci (seed, undercloud, overcloud) | 01:23 |
| adam_g | i gotta run now. cheers | 01:25 |
| *** GMann has joined #openstack-qa | 01:25 | |
| devananda | adam_g: cheers, thanks! | 01:25 |
| *** jyuso has joined #openstack-qa | 01:27 | |
| mtreinish | devananda: so I've got run too, but I want to make sure we've got a direction moving forward | 01:28 |
| *** harlowja_ has quit IRC | 01:28 | |
| devananda | mtreinish: thanks for sticking around to chat | 01:29 |
| devananda | mtreinish: i think we're making progress, though i also think this raised some questions for me about our current tempest coverage | 01:29 |
| *** oomichi has quit IRC | 01:29 | |
| mtreinish | yeah I need to start looking at things a bit more closely too | 01:29 |
| devananda | i think "enable running the api tests o neral hardware" is reasonably achievable | 01:31 |
| devananda | *on real | 01:32 |
| *** harlowja has joined #openstack-qa | 01:32 | |
| mtreinish | yeah I think so too | 01:32 |
| devananda | even if not advisable | 01:32 |
| mtreinish | user beware :) | 01:32 |
| *** oomichi has joined #openstack-qa | 01:33 | |
| *** nati_ueno has quit IRC | 01:40 | |
| openstackgerrit | Brant Knudson proposed a change to openstack/tempest: Add test for GET /v3 https://review.openstack.org/101977 | 01:46 |
| *** malini1 has quit IRC | 01:47 | |
| *** nosnos has joined #openstack-qa | 01:51 | |
| *** nosnos has quit IRC | 01:56 | |
| *** nosnos has joined #openstack-qa | 01:57 | |
| *** morganfainberg_Z is now known as morganfainberg | 02:06 | |
| *** jyuso has quit IRC | 02:18 | |
| *** jyuso has joined #openstack-qa | 02:18 | |
| *** jamielennox is now known as jamielennox|away | 02:22 | |
| *** dims__ has quit IRC | 02:28 | |
| *** openstackgerrit has quit IRC | 02:31 | |
| *** mitz_ has joined #openstack-qa | 02:36 | |
| *** malini1 has joined #openstack-qa | 02:44 | |
| *** dims__ has joined #openstack-qa | 02:54 | |
| *** dims__ has quit IRC | 02:58 | |
| *** harlowja is now known as harlowja_away | 03:04 | |
| *** malini1 has quit IRC | 03:25 | |
| *** nati_ueno has joined #openstack-qa | 03:25 | |
| *** GMann has quit IRC | 03:29 | |
| *** weshay has quit IRC | 03:34 | |
| *** GMann has joined #openstack-qa | 03:37 | |
| *** matsuhashi has quit IRC | 03:42 | |
| *** matsuhashi has joined #openstack-qa | 03:43 | |
| *** matsuhashi has quit IRC | 03:47 | |
| *** nosnos has quit IRC | 03:51 | |
| *** dims__ has joined #openstack-qa | 03:55 | |
| *** Poornima has joined #openstack-qa | 03:55 | |
| *** mmaglana has quit IRC | 03:58 | |
| *** mmaglana has joined #openstack-qa | 03:58 | |
| *** dims__ has quit IRC | 03:59 | |
| *** mmaglana has quit IRC | 04:03 | |
| *** beisner has quit IRC | 04:07 | |
| *** nati_ueno has quit IRC | 04:08 | |
| *** nati_ueno has joined #openstack-qa | 04:09 | |
| *** mlavalle has quit IRC | 04:25 | |
| *** yfried has quit IRC | 04:25 | |
| *** beisner has joined #openstack-qa | 04:27 | |
| *** nati_ueno has quit IRC | 04:29 | |
| *** matsuhashi has joined #openstack-qa | 04:31 | |
| *** Manishanker has joined #openstack-qa | 04:35 | |
| *** nosnos has joined #openstack-qa | 04:37 | |
| *** Sukhdev has joined #openstack-qa | 04:44 | |
| *** SridharG has joined #openstack-qa | 04:52 | |
| raies | devananda: hi | 04:54 |
| *** dims__ has joined #openstack-qa | 04:56 | |
| *** bkopilov has joined #openstack-qa | 04:56 | |
| *** ajc_ has joined #openstack-qa | 04:59 | |
| *** yfried has joined #openstack-qa | 04:59 | |
| *** dims__ has quit IRC | 05:00 | |
| *** Manishanker has quit IRC | 05:05 | |
| *** morganfainberg is now known as morganfainberg_Z | 05:14 | |
| *** psharma has joined #openstack-qa | 05:16 | |
| *** markmcclain has joined #openstack-qa | 05:17 | |
| *** nati_ueno has joined #openstack-qa | 05:17 | |
| *** markmcclain1 has joined #openstack-qa | 05:18 | |
| *** coolsvap|afk is now known as coolsvap | 05:19 | |
| *** markmcclain has quit IRC | 05:22 | |
| *** rwsu has quit IRC | 05:24 | |
| *** nati_ueno has quit IRC | 05:26 | |
| *** ildikov has quit IRC | 05:27 | |
| *** amitpp has joined #openstack-qa | 05:31 | |
| *** tkammer has joined #openstack-qa | 05:33 | |
| *** markmcclain1 has quit IRC | 05:39 | |
| *** amitpp has quit IRC | 05:42 | |
| *** ildikov has joined #openstack-qa | 05:54 | |
| *** Sukhdev has quit IRC | 05:56 | |
| *** dims__ has joined #openstack-qa | 05:56 | |
| *** mmaglana has joined #openstack-qa | 05:58 | |
| *** chandan_kumar has quit IRC | 06:00 | |
| *** dims__ has quit IRC | 06:01 | |
| *** mmaglana has quit IRC | 06:03 | |
| *** afazekas_ has joined #openstack-qa | 06:13 | |
| *** psharma_ has joined #openstack-qa | 06:42 | |
| *** nati_ueno has joined #openstack-qa | 06:42 | |
| *** Poornima|mtg has joined #openstack-qa | 06:42 | |
| *** catohornet has quit IRC | 06:42 | |
| *** kashyap has quit IRC | 06:45 | |
| *** psharma__ has joined #openstack-qa | 06:45 | |
| *** Poornima|afk has joined #openstack-qa | 06:45 | |
| *** psharma has quit IRC | 06:45 | |
| *** Poornima has quit IRC | 06:45 | |
| *** SridharG has quit IRC | 06:46 | |
| *** kashyap has joined #openstack-qa | 06:48 | |
| *** Poornima|mtg has quit IRC | 06:48 | |
| *** psharma_ has quit IRC | 06:49 | |
| *** kashyap` has joined #openstack-qa | 06:52 | |
| *** psharma_ has joined #openstack-qa | 06:52 | |
| *** Poornima|mtg has joined #openstack-qa | 06:52 | |
| *** yrabl has joined #openstack-qa | 06:54 | |
| *** psharma__ has quit IRC | 06:55 | |
| *** kashyap has quit IRC | 06:56 | |
| *** Poornima|afk has quit IRC | 06:56 | |
| *** dims__ has joined #openstack-qa | 06:57 | |
| *** dims__ has quit IRC | 07:04 | |
| *** kashyap` is now known as kashyap | 07:05 | |
| *** jordanP has joined #openstack-qa | 07:14 | |
| *** gfidente has joined #openstack-qa | 07:23 | |
| *** dkorn has quit IRC | 07:33 | |
| *** SridharG has joined #openstack-qa | 07:34 | |
| *** Longgeek has joined #openstack-qa | 07:37 | |
| *** Longgeek has quit IRC | 07:37 | |
| *** Longgeek has joined #openstack-qa | 07:37 | |
| *** ajo|mac has joined #openstack-qa | 07:39 | |
| *** jpich has joined #openstack-qa | 07:50 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 07:53 | |
| *** dkorn has joined #openstack-qa | 07:54 | |
| *** ajo|mac has quit IRC | 07:56 | |
| *** ajo|mac has joined #openstack-qa | 07:57 | |
| *** matsuhashi has quit IRC | 07:57 | |
| *** matsuhashi has joined #openstack-qa | 07:58 | |
| *** ajo|mac has quit IRC | 07:58 | |
| *** mmaglana has joined #openstack-qa | 07:58 | |
| *** tosky has joined #openstack-qa | 07:59 | |
| *** evgenyf has joined #openstack-qa | 08:00 | |
| *** mmaglana has quit IRC | 08:03 | |
| *** matsuhashi has quit IRC | 08:05 | |
| *** jlibosva has joined #openstack-qa | 08:05 | |
| kashyap | afazekas, afazekas_ When you're around -- are you able to launch DevStack instances w/ today's git on F20? | 08:06 |
| kashyap | Details here -- http://lists.openstack.org/pipermail/openstack-dev/2014-June/038619.html | 08:07 |
| * kashyap off to make some lunch | 08:07 | |
| *** kashyap is now known as kashyap_lunch | 08:07 | |
| *** qba73 has joined #openstack-qa | 08:08 | |
| afazekas_ | kashyap: yes | 08:09 |
| afazekas_ | looks like you do not have some yaml dev lib | 08:10 |
| *** matsuhashi has joined #openstack-qa | 08:10 | |
| yfried | eezhova: here? | 08:10 |
| *** qba73 has quit IRC | 08:15 | |
| *** qba73 has joined #openstack-qa | 08:15 | |
| afazekas_ | kashyap: do you know which module does it wanted to rebuild ? Do you have libyaml ? | 08:16 |
| afazekas_ | kashyap: did you used a cloud image ? | 08:17 |
| *** matsuhashi has quit IRC | 08:17 | |
| *** matsuhashi has joined #openstack-qa | 08:18 | |
| kashyap_lunch | afazekas_, I'm still making lunch. . . | 08:24 |
| kashyap_lunch | afazekas_, Use cloud image for? As a devstack VM? | 08:24 |
| kashyap_lunch | Or Nova guest? | 08:24 |
| * afazekas_ some items on my endless todo list: ofagent become broken around 20th; n-cell fails more tests with qpid than with rabbit; the lvm commands prints warnings if you do not start/restart lvmetad (otherwise working | 08:24 | |
| kashyap_lunch | I'm using the cloud image as Nova guest. | 08:24 |
| afazekas_ | kashyap_lunch: as devstack vm, it probbaly be default has the libyaml | 08:25 |
| afazekas_ | kashyap_lunch: Again did you installed the libyaml ? do you know which package was tried to be compiled | 08:26 |
| *** qba73 has quit IRC | 08:26 | |
| kashyap_lunch | I created the DevStack VM like that: | 08:27 |
| kashyap_lunch | $ virt-builder fedora-20 -o devstack.qcow2 --update --selinux-relabel --format qcow2 --size 60G | 08:27 |
| kashyap_lunch | afazekas, I'll check in a few, otherwise things will burn on the stove :-) | 08:27 |
| kashyap_lunch | afazekas_, Grep for "compile" here -- http://kashyapc.fedorapeople.org/temp/devstacklog-25JUN2014.txt | 08:29 |
| afazekas_ | PyYAML-3.10-9.fc20.x86_64 was installed | 08:30 |
| *** matsuhashi has quit IRC | 08:31 | |
| *** Ju_ has quit IRC | 08:34 | |
| afazekas_ | PyYAML>=3.1.0 in the global req | 08:35 |
| afazekas_ | PyYAML-3.10-9.fc20.x86_64 is on the cloud image | 08:36 |
| afazekas_ | kashyap_lunch: you can try to add PyYAML or the build time deps for PyYAML at devstack/files/*/general | 08:38 |
| *** andreaf has joined #openstack-qa | 08:39 | |
| *** mkollaro has joined #openstack-qa | 08:45 | |
| *** nosnos has quit IRC | 08:45 | |
| *** eezhova has quit IRC | 08:46 | |
| kashyap_lunch | Thanks for the suggestion. | 08:48 |
| afazekas_ | kashyap_lunch: cloud-init depends on PyYAML so it should be installed on every cloud-image at the 0th second, I guess your image is not a cloud image | 08:49 |
| kashyap_lunch | afazekas_, Right, it is not. I don't see PyYAML on the DevStack VM, will manually install and rerun it | 08:49 |
| *** openstackgerrit has joined #openstack-qa | 08:53 | |
| *** eezhova has joined #openstack-qa | 08:57 | |
| openstackgerrit | Sridhar Gaddam proposed a change to openstack/qa-specs: Spec for IPv6 API testing parity in tempest https://review.openstack.org/102468 | 08:57 |
| *** dmorita has quit IRC | 09:02 | |
| *** dims has joined #openstack-qa | 09:02 | |
| *** tkammer has quit IRC | 09:04 | |
| *** dims has quit IRC | 09:08 | |
| *** kashyap_lunch is now known as kashyap | 09:10 | |
| kashyap | afazekas_, Yes, manually installing & rerunning ./stack.sh of-course removes that compile error | 09:12 |
| openstackgerrit | wingwj proposed a change to openstack/tempest: Add negative tests for server_group https://review.openstack.org/102472 | 09:14 |
| *** jyuso has quit IRC | 09:18 | |
| *** tkammer has joined #openstack-qa | 09:21 | |
| *** amotoki has quit IRC | 09:24 | |
| kashyap | I wonder why DevStack install the package if it didn't find it on the system. /me goes to look at source | 09:28 |
| *** harikav has joined #openstack-qa | 09:29 | |
| *** evgenyf has quit IRC | 09:30 | |
| *** dANOKELOFF has joined #openstack-qa | 09:31 | |
| dANOKELOFF | Hi, anyone can help me with this error please ? https://gist.github.com/anonymous/eb6880e973b2baf12066 | 09:31 |
| *** nati_ueno has quit IRC | 09:31 | |
| *** safchain has joined #openstack-qa | 09:32 | |
| *** nosnos has joined #openstack-qa | 09:36 | |
| *** matsuhashi has joined #openstack-qa | 09:36 | |
| *** jlanoux has joined #openstack-qa | 09:41 | |
| *** nihilifer has joined #openstack-qa | 09:43 | |
| *** nihilifer has left #openstack-qa | 09:44 | |
| *** evgenyf has joined #openstack-qa | 09:45 | |
| afazekas_ | kashyap: similar trick can be used with the f20 image if you would like to use a cloud image outside to a cloud: https://qa.ubuntu.com/2012/06/19/local-ubuntu-vm-provisioning-with-cloud-init/ | 09:46 |
| afazekas_ | basically you need to create a cdrom image which contains your ssh public key | 09:47 |
| kashyap | afazekas_, Yes, I already use something like -- http://rwmj.wordpress.com/2013/12/10/creating-a-cloud-init-config-disk-for-non-cloud-boots/ | 09:49 |
| kashyap | Thanks for the URL | 09:49 |
| afazekas_ | BTW: now possible to build images inside open-stack, without the kexec trick .. | 09:49 |
| *** dANOKELO_ has joined #openstack-qa | 09:52 | |
| *** dANOKELOFF has quit IRC | 09:53 | |
| kashyap | Using disk-image-builder? | 09:54 |
| kashyap | afazekas_, ^ | 09:54 |
| *** matsuhashi has quit IRC | 09:54 | |
| kashyap | For some reason I still find virt-builder _way_ way cleaner and easier for some reason. All my setups use that | 09:55 |
| *** matsuhashi has joined #openstack-qa | 09:55 | |
| afazekas_ | disk-image-builder: no , simply using ks | 09:55 |
| afazekas_ | kashyap: I just read in the news ARMv8 is coming.. http://www.cavium.com/newsevents_Cavium_Cavium_Expands_Collaboration_with_Fedora_Project_to_Accelerate_ARMv8_Software_Ecosystem.html | 09:56 |
| *** matsuhas_ has joined #openstack-qa | 09:57 | |
| kashyap | afazekas_, Ah, I use Kickstart-based auto install all the time - https://github.com/kashyapc/virt-scripts/blob/master/create-guest-qcow2.bash | 09:57 |
| *** matsuhashi has quit IRC | 09:57 | |
| kashyap | afazekas, Hm, I'm still yet to find some reasonable use case where I can use ARM (along w/ virt. I also see upstream KVM has tons of patches for ARM). If you're more curious, check out Rich Jone's blog post series on ARM | 09:58 |
| openstackgerrit | ajay proposed a change to openstack/tempest: Improve neutron CLI existing tests https://review.openstack.org/101707 | 10:05 |
| openstackgerrit | ajay proposed a change to openstack/tempest: Improve nova CLI existing tests https://review.openstack.org/101767 | 10:07 |
| *** yamahata has quit IRC | 10:08 | |
| openstackgerrit | ajay proposed a change to openstack/tempest: Improve cinder CLI existing tests https://review.openstack.org/101694 | 10:10 |
| openstackgerrit | ajay proposed a change to openstack/tempest: Add three new CLI tests of sahara https://review.openstack.org/100784 | 10:11 |
| *** sabeen has quit IRC | 10:15 | |
| *** jpich has quit IRC | 10:20 | |
| *** jpich has joined #openstack-qa | 10:20 | |
| *** matsuhas_ has quit IRC | 10:28 | |
| *** matsuhashi has joined #openstack-qa | 10:28 | |
| *** amitpp has joined #openstack-qa | 10:28 | |
| openstackgerrit | Sridhar Gaddam proposed a change to openstack/tempest: Improve IPV6 parity in Security Group testcases https://review.openstack.org/94130 | 10:30 |
| *** matsuhashi has quit IRC | 10:37 | |
| *** matsuhashi has joined #openstack-qa | 10:37 | |
| *** GMann has quit IRC | 10:41 | |
| *** gszasz has joined #openstack-qa | 10:46 | |
| *** Manishanker has joined #openstack-qa | 10:49 | |
| openstackgerrit | Kashyap Chamarthy proposed a change to openstack-dev/devstack: Add PyYAML dependency for Fedora-based distros https://review.openstack.org/102490 | 10:55 |
| *** malini1 has joined #openstack-qa | 10:59 | |
| *** Manishanker has quit IRC | 11:01 | |
| *** dims has joined #openstack-qa | 11:03 | |
| *** dims has quit IRC | 11:08 | |
| *** coolsvap is now known as coolsvap|afk | 11:11 | |
| *** yamahata has joined #openstack-qa | 11:15 | |
| *** dANOKELO_ has quit IRC | 11:18 | |
| *** dims_ has joined #openstack-qa | 11:19 | |
| openstackgerrit | nayna-patel proposed a change to openstack/tempest: Adds new v3 projects tests to Keystone https://review.openstack.org/93877 | 11:21 |
| afazekas_ | eezhova: ping | 11:21 |
| eezhova | afazekas: pong | 11:22 |
| *** dims__ has joined #openstack-qa | 11:22 | |
| afazekas_ | eezhova: can you change the nc command usage to a similar way as I mentioned ? | 11:23 |
| eezhova | afazekas: sure, I'm working on it right now | 11:23 |
| afazekas_ | cool | 11:24 |
| *** dims_ has quit IRC | 11:25 | |
| eezhova | afazekas: I'll update the path as soon as I test it enough times | 11:25 |
| eezhova | *the patch | 11:26 |
| *** malini1 has quit IRC | 11:34 | |
| *** yfried_ has joined #openstack-qa | 11:35 | |
| *** yfried has quit IRC | 11:35 | |
| *** SridharG has quit IRC | 11:36 | |
| *** raies has quit IRC | 11:38 | |
| *** Manishanker has joined #openstack-qa | 11:45 | |
| kashyap | afazekas_, What would be the impact if I see Jenkins failures for -- "check-tempest-dsvm-neutron-full (non-voting)" | 11:51 |
| kashyap | Do I spend time debugging it? | 11:51 |
| *** mwagner_lap has quit IRC | 11:53 | |
| afazekas_ | kashyap: you can, but usually the failures form the voting ones are more import | 11:53 |
| afazekas_ | kashyap: I would like to see the full neutron job voting ASAP | 11:54 |
| kashyap | afazekas_, All voting ones are successful. It's a trivial change -- https://review.openstack.org/#/c/102490/ | 11:54 |
| kashyap | I was wondering if that's (adding the explicit PyYAML dep) the right thing to do | 11:55 |
| afazekas_ | check-tempest-dsvm-full is failed and voting | 11:55 |
| kashyap | afazekas_, Oops, yeah, was already looking at the logs | 11:55 |
| kashyap | Related to -- "ERROR: testr-full: commands failed" | 11:55 |
| kashyap | Hmm, seems like a pretty_tox.sh invocation error | 11:56 |
| kashyap | http://logs.openstack.org/90/102490/1/check/check-tempest-dsvm-neutron-full/bad3225/console.html#_2014-06-25_11_38_57_753 | 11:57 |
| afazekas_ | kashyap: BTW: you can try to look up the equivalent package name for other distro , python-yaml seams to one used on ubuntu and opensuse | 11:58 |
| *** adalbas has joined #openstack-qa | 11:58 | |
| afazekas_ | kashyap: https://bugs.launchpad.net/nova/+bug/1320617 | 11:59 |
| kashyap | afazekas_, Okay, let me check. . | 11:59 |
| kashyap | afazekas_, What other distros are you referring to? python-yaml it is -- for both Ubuntu & OpenSuse | 12:02 |
| kashyap | afazekas_, And, what about the above bug? I don't see it relevant to the "pretty_tox.sh" invocation failure that I'm seeing. Or was that just an FYI? | 12:04 |
| afazekas_ | http://logs.openstack.org/90/102490/1/check/check-tempest-dsvm-neutron-full/bad3225/console.html#_2014-06-25_11_38_57_714 | 12:05 |
| afazekas_ | Details: (ListImageFiltersTestJSON:setUpClass) Image 462a64c3-ceb6-4816-a8bc-6788d08eb8c6 failed to reach ACTIVE status within the required time (196 s). Current status: SAVING. | 12:05 |
| kashyap | Yep, noticed that | 12:06 |
| kashyap | I'm not sure where to start debugging this - ERROR: InvocationError: '/bin/bash tools/pretty_tox.sh (?!.*\\[.*\\bslow\\b.*\\])(^tempest\\.(api|scenario|thirdparty|cli)) --concurrency=4' | 12:06 |
| afazekas_ | it is #1320617 | 12:06 |
| * kashyap goes to read it with more focus | 12:06 | |
| kashyap | Aha, yes. just grepped it | 12:08 |
| kashyap | afazekas_, You don't mention the tox failures in the bug? | 12:08 |
| afazekas_ | kashyap: because it fails if any of the test cases fails | 12:10 |
| kashyap | afazekas_, I see. Still learning debugging Gate stuff. | 12:11 |
| kashyap | One more -- I wonder if I should revise the patch - https://review.openstack.org/#/c/102490/ and add the python-yaml for other distributions (apt and rpms-suse too)? | 12:12 |
| *** weshay has joined #openstack-qa | 12:17 | |
| *** dkranz has joined #openstack-qa | 12:18 | |
| *** amitpp has quit IRC | 12:19 | |
| *** amitpp has joined #openstack-qa | 12:19 | |
| *** amitpp has quit IRC | 12:22 | |
| *** ajc_ has quit IRC | 12:23 | |
| *** dkranz has quit IRC | 12:27 | |
| *** yfried_ has quit IRC | 12:29 | |
| *** dims__ has quit IRC | 12:31 | |
| *** dims__ has joined #openstack-qa | 12:31 | |
| openstackgerrit | Elena Ezhova proposed a change to openstack/tempest: Correct nc command in test_load_balancer_basic https://review.openstack.org/98065 | 12:37 |
| *** dkranz has joined #openstack-qa | 12:39 | |
| *** tkammer has quit IRC | 12:41 | |
| *** dANOKELOFF has joined #openstack-qa | 12:42 | |
| *** yfried has joined #openstack-qa | 12:43 | |
| *** ekhugen has joined #openstack-qa | 12:50 | |
| *** SridharG has joined #openstack-qa | 12:50 | |
| *** dustins has joined #openstack-qa | 12:54 | |
| *** amitpp has joined #openstack-qa | 12:58 | |
| *** matsuhashi has quit IRC | 12:59 | |
| *** matsuhashi has joined #openstack-qa | 12:59 | |
| *** tkammer has joined #openstack-qa | 12:59 | |
| *** enikanorov_ has quit IRC | 13:00 | |
| *** enikanorov_ has joined #openstack-qa | 13:00 | |
| *** yfried has quit IRC | 13:04 | |
| *** matsuhashi has quit IRC | 13:04 | |
| *** safchain has quit IRC | 13:06 | |
| openstackgerrit | Dina Belova proposed a change to openstack/qa-specs: Add spec for the bp:add-ceilometer-scenario-tests https://review.openstack.org/97253 | 13:06 |
| openstackgerrit | Dina Belova proposed a change to openstack/qa-specs: Add spec for the bp:add-basic-ceilometer-tests https://review.openstack.org/97252 | 13:06 |
| *** safchain has joined #openstack-qa | 13:06 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: add fingerprint for Dump was interrupted bug https://review.openstack.org/102517 | 13:08 |
| *** matsuhashi has joined #openstack-qa | 13:10 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: add fingerprint for Dump was interrupted bug https://review.openstack.org/102517 | 13:11 |
| *** Manishanker has quit IRC | 13:12 | |
| *** mriedem has joined #openstack-qa | 13:15 | |
| *** mwagner_lap has joined #openstack-qa | 13:16 | |
| *** salv-orlando_ has joined #openstack-qa | 13:18 | |
| *** salv-orlando has quit IRC | 13:20 | |
| *** salv-orlando_ is now known as salv-orlando | 13:20 | |
| *** harikav has quit IRC | 13:21 | |
| *** malini1 has joined #openstack-qa | 13:26 | |
| *** Obulpathi has joined #openstack-qa | 13:28 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: add fingerprint for Dump was interrupted bug https://review.openstack.org/102517 | 13:30 |
| *** amitpp has quit IRC | 13:30 | |
| openstackgerrit | Brant Knudson proposed a change to openstack/tempest: Add test for GET /v3 https://review.openstack.org/101977 | 13:30 |
| *** Obulpathi has quit IRC | 13:32 | |
| *** Obulpathi has joined #openstack-qa | 13:33 | |
| *** matsuhashi has quit IRC | 13:33 | |
| *** Obulpathi has quit IRC | 13:33 | |
| *** matsuhashi has joined #openstack-qa | 13:33 | |
| *** Obulpathi has joined #openstack-qa | 13:35 | |
| *** markmcclain has joined #openstack-qa | 13:36 | |
| openstackgerrit | Dina Belova proposed a change to openstack/qa-specs: Add spec for the bp:add-ceilometer-scenario-tests https://review.openstack.org/97253 | 13:36 |
| *** Obulpathi has quit IRC | 13:36 | |
| *** Obulpathi has joined #openstack-qa | 13:37 | |
| *** markmcclain1 has joined #openstack-qa | 13:37 | |
| *** matsuhashi has quit IRC | 13:38 | |
| *** openstackrecheck has joined #openstack-qa | 13:38 | |
| *** markmcclain1 has quit IRC | 13:38 | |
| *** markmcclain has quit IRC | 13:40 | |
| *** psharma_ has quit IRC | 13:43 | |
| *** nosnos has quit IRC | 13:43 | |
| *** coolsvap|afk is now known as coolsvap | 13:43 | |
| *** matsuhashi has joined #openstack-qa | 13:44 | |
| *** matsuhashi has quit IRC | 13:44 | |
| *** matsuhashi has joined #openstack-qa | 13:45 | |
| malini1 | dkranz: Can you take a look at https://review.openstack.org/#/c/94944/ when you get a chance? | 13:49 |
| *** matsuhashi has quit IRC | 13:49 | |
| malini1 | I have addressed you review comments from the previous patch set | 13:49 |
| *** yfried has joined #openstack-qa | 13:53 | |
| *** yfried has quit IRC | 13:54 | |
| *** yfried has joined #openstack-qa | 13:54 | |
| openstackrecheck | Console logs not available after 13:35s for check-grenade-dsvm-partial-ncpu 98065,2,0ac0e59 | 13:55 |
| *** nati_ueno has joined #openstack-qa | 13:55 | |
| *** rbak has joined #openstack-qa | 14:05 | |
| *** ildikov has quit IRC | 14:06 | |
| openstackrecheck | Console logs not available after 13:21s for gate-sahara-pep8 101845,3,0732dde | 14:08 |
| *** nikhil___ has joined #openstack-qa | 14:10 | |
| openstackgerrit | Stanislaw Pitucha proposed a change to openstack-dev/devstack: Don't try to regenerate existing ssl certificates https://review.openstack.org/102535 | 14:11 |
| nikhil___ | mtreinish: o/ | 14:11 |
| *** zhiyan has joined #openstack-qa | 14:12 | |
| mtreinish | nikhil___: ok, so you want to add glance tests for image data? Is that just uploading a binary data and verifying you get the right thing back? | 14:12 |
| nikhil___ | mtreinish: heh, to tell you the truth, I'm not completely sure what needs to happen. However, yes from whatever info was recieved that seems accurate. | 14:13 |
| nikhil___ | mtreinish: trying to find the link to the wiki which specifies the requirements | 14:14 |
| nikhil___ | was hoping someone here would point me out :) | 14:14 |
| * nikhil___ googling | 14:14 | |
| mtreinish | nikhil___: this one?: https://wiki.openstack.org/wiki/Governance/TechnicalCommittee/Glance_Gap_Coverage | 14:14 |
| nikhil___ | mtreinish: yeah! Thank you. | 14:15 |
| *** nati_ueno has quit IRC | 14:16 | |
| mtreinish | yeah so it should be pretty simple, we just need to add a couple api tests to tempest.api.image that instead of using random strings for the image just send random binary data | 14:16 |
| mtreinish | because in the test we won't have access to real images | 14:16 |
| mtreinish | or at least that's how I interpret it | 14:17 |
| nikhil___ | (side-questions for later, so that I don't forget to ask) am assuming high-bit-set refers to high-order bit set | 14:17 |
| mtreinish | yeah I'm assuming so | 14:17 |
| nikhil___ | mtreinish: makes sense, so random binary data should be "produced" on the local disk? | 14:18 |
| nikhil___ | or download from cirros links? | 14:18 |
| nikhil___ | (or something of that order) | 14:18 |
| mtreinish | so I think downloading soemthing locally and uploading that is probably not what we want to do | 14:19 |
| mtreinish | downloads are prone to failure in gating | 14:19 |
| mtreinish | it doesn't have to be a real image | 14:19 |
| openstackgerrit | Henry Gessau proposed a change to openstack-dev/devstack: Don't install openvswitch-datapath-dkms on newer kernels https://review.openstack.org/102284 | 14:19 |
| mtreinish | I think it's more just a test of sending and recieving arbitrary data | 14:19 |
| mtreinish | nikhil___: what we have now: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/image/v2/test_images.py#n55 | 14:20 |
| nikhil___ | mtreinish: makes sense, do we have access to a /*/tmp/ folder where data can be stored? (sorry, for being noobish to gating) | 14:20 |
| *** Poornima|mtg has quit IRC | 14:20 | |
| *** sileht has quit IRC | 14:20 | |
| * nikhil___ looking | 14:20 | |
| mtreinish | yeah you can make a tempfile if needed, but we can just do it in memory I think | 14:20 |
| *** hemanth_ has joined #openstack-qa | 14:21 | |
| sdague | nikhil___: we actually have a cirros image | 14:21 |
| mtreinish | sdague: isn't it already in glance though? | 14:21 |
| *** gszasz has quit IRC | 14:21 | |
| nikhil___ | mtreinish: gotcha, just trying to confirm for performance purposes as the new rally tests keep a stored image for some of the tests | 14:21 |
| sdague | mtreinish: sure, there is no reason why it couldn't be a second time | 14:21 |
| sdague | either pull / push | 14:21 |
| openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-full 89897,39,1f45503 | 14:22 |
| nikhil___ | sdague: nice, is that /var/lib/images folder like devstack ? | 14:22 |
| sdague | or just have specifying location for it | 14:22 |
| mtreinish | sdague: yeah I guess that would work too, pull it from glance and upload a new copy of the image | 14:22 |
| nikhil___ | hmm, makes sense | 14:22 |
| sdague | because we make you specify a valid image somewhere by id | 14:22 |
| sdague | so pull / push on it would be good | 14:23 |
| nikhil___ | mtreinish: sdague : only thing concerning is that pulling tests failing might lead to pushing failing. | 14:23 |
| sdague | nikhil___: yeh, but if you can't pull an image from glance... that seems like a problem | 14:23 |
| sdague | because that's kind of it's job | 14:23 |
| *** Poornima|mtg has joined #openstack-qa | 14:23 | |
| nikhil___ | :D true that | 14:23 |
| *** sileht has joined #openstack-qa | 14:23 | |
| zhiyan | nikhil___: can we use v2 direct url to access that image directly instead of pull it | 14:23 |
| nikhil___ | zhiyan: good idea, may be we can separate those tests out for v2 as we would need v1 compliant | 14:24 |
| sdague | zhiyan: that won't work from outside a cloud though, right | 14:24 |
| sdague | don't assume tempest is running on a cloud node, even though we do that in the gate | 14:24 |
| mtreinish | yeah, doing it through the api would probably be for the best | 14:25 |
| nikhil___ | mtreinish: sdague: zhiyan : would love it to use existing data from somewhere if it's stored for common use. If that's not the case, will try to see options.. | 14:26 |
| sdague | nikhil___: yeh, we could require specifying the image in the config as well | 14:26 |
| nikhil___ | (my failing sincere attempt of keeping things independent) | 14:26 |
| openstackgerrit | Matt Riedemann proposed a change to openstack/qa-specs: Check minimum version for CLI tests https://review.openstack.org/99984 | 14:27 |
| nikhil___ | sdague: yeah, that would be nice. | 14:27 |
| zhiyan | nikhil___: as sdague said probably we can ask tester set a image path in config file? | 14:27 |
| zhiyan | or a image-id? and pull/push | 14:28 |
| *** mlavalle has joined #openstack-qa | 14:28 | |
| mtreinish | zhiyan: we already have 2 image ids in the config | 14:28 |
| nikhil___ | zhiyan: assuming you'r referring to tester as a module and not a real person ( ? ) | 14:28 |
| sdague | zhiyan: yeh, honesty, the more I think about it, I think a path is good | 14:28 |
| nikhil___ | sdague: +1 | 14:29 |
| zhiyan | sdague: cool. path option+push, and keeping things independent | 14:29 |
| mtreinish | sdague: that is basically what we already do for the image location tests. Specify an http address for a remote image | 14:29 |
| mtreinish | so I think that's fine | 14:29 |
| nikhil___ | mtreinish: do we test for all the glance stores, do you know? | 14:29 |
| mtreinish | nikhil___: In the gate? | 14:30 |
| *** ildikov has joined #openstack-qa | 14:30 | |
| mtreinish | nikhil___: the answer is probably no either way | 14:31 |
| openstackgerrit | Stanislaw Pitucha proposed a change to openstack-dev/devstack: Do a proper tls/ca cleanup in unstack https://review.openstack.org/102542 | 14:31 |
| nikhil___ | mtreinish: yeah, if we test filesystem store for glance then it would be easier to get some image data | 14:31 |
| openstackgerrit | Henry Gessau proposed a change to openstack-dev/devstack: Don't install openvswitch-datapath-dkms on newer kernels https://review.openstack.org/102284 | 14:32 |
| * nikhil___ will try to spend some time looking into tempest tests for glance and not bug people here | 14:32 | |
| mtreinish | nikhil___: honestly the glance tests are pretty simple/limited in tempest | 14:34 |
| nikhil___ | cool | 14:35 |
| openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-full 91011,32,1e74b31 | 14:35 |
| *** gszasz has joined #openstack-qa | 14:35 | |
| nikhil___ | Thanks mtreinish sdague zhiyan, appreciate it! | 14:36 |
| zhiyan | nikhil___: sdague mtreinish thanks! | 14:36 |
| mtreinish | nikhil___: heh, I'm not sure I'd say having limited testing is cool :) | 14:37 |
| mtreinish | sure, np | 14:37 |
| *** sabeen has joined #openstack-qa | 14:40 | |
| nikhil___ | mtreinish: heh, true that. Was just remarking to finally know about the testing status vs. it's coverage :=] Hope to get it better soon. | 14:41 |
| *** morganfainberg_Z is now known as morganfainberg | 14:42 | |
| *** zhiyan is now known as zhiyan_ | 14:43 | |
| openstackgerrit | Andrey Kurilin proposed a change to openstack/qa-specs: Automate tempest config generation https://review.openstack.org/94473 | 14:47 |
| openstackrecheck | Console logs not available after 13:23s for gate-cinder-python26 97091,34,c12850a | 14:48 |
| *** markmcclain has joined #openstack-qa | 14:48 | |
| *** rwsu has joined #openstack-qa | 14:49 | |
| openstackgerrit | Joseph Lanoux proposed a change to openstack/tempest: Add skip decorators in test_servers_negative https://review.openstack.org/102546 | 14:50 |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-dev/devstack: Use the apache 2.4 ErrorLogFormat directive https://review.openstack.org/102425 | 14:51 |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Remove old queries for fixed bugs https://review.openstack.org/102547 | 14:52 |
| *** oomichi has quit IRC | 14:52 | |
| mriedem | jogo: so heat unit tests seem to be f'ed | 14:58 |
| *** Obulpathi has quit IRC | 14:59 | |
| *** mwagner_lap has quit IRC | 15:00 | |
| *** Obulpathi has joined #openstack-qa | 15:00 | |
| openstackgerrit | enikanorov proposed a change to openstack-infra/elastic-recheck: Add query for bug #1332500 https://review.openstack.org/102551 | 15:00 |
| mriedem | jogo: nevermind, they fixed it already | 15:02 |
| *** yrabl has quit IRC | 15:02 | |
| openstackrecheck | Console logs not available after 13:22s for gate-tempest-dsvm-postgres-full 95723,8,06afca6 | 15:02 |
| *** tkammer has quit IRC | 15:02 | |
| *** Poornima|mtg has quit IRC | 15:05 | |
| *** dkranz has quit IRC | 15:05 | |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for heat bug 1333556 https://review.openstack.org/102554 | 15:09 |
| *** evgenyf has quit IRC | 15:10 | |
| *** malini1 has quit IRC | 15:11 | |
| *** malini1 has joined #openstack-qa | 15:12 | |
| *** malini1 has quit IRC | 15:12 | |
| *** malini1 has joined #openstack-qa | 15:13 | |
| openstackrecheck | Console logs not available after 13:21s for gate-nova-python26 102522,1,632c576 | 15:15 |
| *** mwagner_lap has joined #openstack-qa | 15:16 | |
| *** dkranz has joined #openstack-qa | 15:18 | |
| openstackgerrit | enikanorov proposed a change to openstack-infra/elastic-recheck: Add query for bug #1332500 https://review.openstack.org/102551 | 15:21 |
| *** andreaf_ has joined #openstack-qa | 15:27 | |
| openstackgerrit | enikanorov proposed a change to openstack-infra/elastic-recheck: Add query for neutron bug 1334109 https://review.openstack.org/102558 | 15:28 |
| *** andreaf_ has quit IRC | 15:28 | |
| openstackrecheck | Console logs not available after 13:22s for gate-horizon-python26 91118,32,f85cdbd | 15:28 |
| *** aimon_ has joined #openstack-qa | 15:31 | |
| openstackgerrit | Joseph Lanoux proposed a change to openstack/tempest: Add skip decorators in test_servers_negative https://review.openstack.org/102546 | 15:34 |
| *** ajo|mac has joined #openstack-qa | 15:39 | |
| openstackgerrit | enikanorov proposed a change to openstack-infra/elastic-recheck: Add query for neutron bug 1334109 https://review.openstack.org/102558 | 15:40 |
| openstackgerrit | enikanorov proposed a change to openstack-infra/elastic-recheck: Add query for bug #1332500 https://review.openstack.org/102551 | 15:41 |
| openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-postgres-full 97996,3,41985b5 | 15:42 |
| *** Sukhdev has joined #openstack-qa | 15:42 | |
| *** mmaglana has joined #openstack-qa | 15:42 | |
| *** ajo|mac has quit IRC | 15:46 | |
| *** dkranz has quit IRC | 15:49 | |
| openstackgerrit | Mike Perez proposed a change to openstack-dev/devstack: Default cinder enable_v1_api to true https://review.openstack.org/102568 | 15:53 |
| openstackrecheck | Console logs not available after 13:41s for gate-trove-pep8 102526,1,0389fd0 | 15:56 |
| *** Obulpathi has quit IRC | 15:57 | |
| *** yamahata has quit IRC | 15:58 | |
| *** dANOKELOFF has quit IRC | 15:59 | |
| *** dkranz has joined #openstack-qa | 16:00 | |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/elastic-recheck: Add query for nova bug 1334345 https://review.openstack.org/102571 | 16:05 |
| *** jlanoux has quit IRC | 16:06 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/devstack-gate: add network_sanity_check https://review.openstack.org/101976 | 16:07 |
| *** malini1 has quit IRC | 16:08 | |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-full 95570,1,174570b | 16:09 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for heat bug 1333556 https://review.openstack.org/102554 | 16:10 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Clarifies documentation about wildcards in queries https://review.openstack.org/102573 | 16:10 |
| *** malini|afk has joined #openstack-qa | 16:11 | |
| *** malini has quit IRC | 16:13 | |
| *** malini|afk is now known as malini | 16:13 | |
| *** aimon_ has quit IRC | 16:15 | |
| *** cbader has joined #openstack-qa | 16:16 | |
| openstackgerrit | Stanislaw Pitucha proposed a change to openstack-dev/devstack: Always check keystone using http https://review.openstack.org/99433 | 16:17 |
| openstackgerrit | Stanislaw Pitucha proposed a change to openstack-dev/devstack: Do a proper tls/ca cleanup in unstack https://review.openstack.org/102542 | 16:17 |
| openstackrecheck | Console logs not available after 13:21s for gate-ceilometer-python26 102492,2,e9acc37 | 16:22 |
| *** openstackrecheck has quit IRC | 16:23 | |
| *** openstackrecheck has joined #openstack-qa | 16:23 | |
| *** dkorn has quit IRC | 16:24 | |
| *** amotoki has joined #openstack-qa | 16:26 | |
| *** afazekas_ has quit IRC | 16:26 | |
| *** mmaglana has quit IRC | 16:31 | |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full 99276,3,056cad8 | 16:37 |
| *** malini is now known as malini|afk | 16:38 | |
| *** aimon_ has joined #openstack-qa | 16:43 | |
| *** aimon_ has quit IRC | 16:44 | |
| *** aimon_ has joined #openstack-qa | 16:44 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/devstack-gate: double up [ for safety https://review.openstack.org/100728 | 16:49 |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-full 89897,39,c5aed9f | 16:51 |
| *** malini|afk is now known as malini | 16:51 | |
| *** jordanP has quit IRC | 16:52 | |
| *** mmaglana has joined #openstack-qa | 16:52 | |
| *** malini is now known as malini|afk | 16:53 | |
| *** mwagner_lap has quit IRC | 16:54 | |
| *** malini|afk is now known as malini | 16:55 | |
| *** safchain has quit IRC | 16:56 | |
| *** jpich has quit IRC | 16:59 | |
| openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-postgres-full 102347,1,4fe6eef | 17:04 |
| jogo | mriedem: isn't it nice when that happens | 17:08 |
| jogo | mriedem: although it looks like nova is the only unit tests failing in the gate | 17:08 |
| *** malini is now known as malini|afk | 17:13 | |
| *** harlowja_away is now known as harlowja | 17:13 | |
| *** malini|afk is now known as malini | 17:15 | |
| *** jaypipes has joined #openstack-qa | 17:17 | |
| *** ChanServ sets mode: +o jaypipes | 17:17 | |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full 100058,2,9992bcc | 17:17 |
| *** mkollaro has quit IRC | 17:19 | |
| *** gszasz has quit IRC | 17:19 | |
| *** jlibosva has quit IRC | 17:21 | |
| *** mkollaro has joined #openstack-qa | 17:25 | |
| *** danfinch has joined #openstack-qa | 17:27 | |
| openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-postgres-full 92124,6,3ed4acd | 17:31 |
| *** dkorn has joined #openstack-qa | 17:36 | |
| *** johnthetubaguy is now known as zz_johnthetubagu | 17:41 | |
| *** Sukhdev has quit IRC | 17:42 | |
| *** jaypipes has quit IRC | 17:44 | |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-full 92125,6,6c6b5a4 | 17:44 |
| *** jaypipes has joined #openstack-qa | 17:44 | |
| *** ChanServ sets mode: +o jaypipes | 17:44 | |
| *** atiwari has joined #openstack-qa | 17:48 | |
| *** dkorn has quit IRC | 17:53 | |
| *** Obulpathi has joined #openstack-qa | 17:54 | |
| openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-full 102480,2,ab2d48f | 17:58 |
| mrodden | sdague: the bash8 rename is schedule for this sat. at 1500 UTC btw | 17:59 |
| mrodden | i had/have an experimental change for devstack to use tox but it (obviously) doesn't find all the files yet | 17:59 |
| sdague | mrodden: cool | 17:59 |
| sdague | mrodden: I actually have tox changes up for all the shell projects | 18:00 |
| sdague | well, devstack, d-g, grenade (which I merged) | 18:00 |
| *** nati_ueno has joined #openstack-qa | 18:00 | |
| *** Sukhdev has joined #openstack-qa | 18:02 | |
| mrodden | oh i saw d-g | 18:02 |
| mrodden | i'll look again | 18:02 |
| openstackrecheck | Console logs not available after 13:21s for gate-nova-pep8 102563,1,2caa055 | 18:11 |
| *** coolsvap is now known as coolsvap|afk | 18:11 | |
| *** afazekas_ has joined #openstack-qa | 18:13 | |
| *** tosky has quit IRC | 18:17 | |
| openstackrecheck | Console logs not available after 13:22s for gate-trove-pep8 102566,1,5e796e6 | 18:24 |
| *** dkorn has joined #openstack-qa | 18:25 | |
| *** mwagner_lap has joined #openstack-qa | 18:27 | |
| *** gszasz has joined #openstack-qa | 18:27 | |
| openstackgerrit | Doug Hellmann proposed a change to openstack-dev/devstack: Add explicit libkrb5-dev dependency for keystone https://review.openstack.org/102600 | 18:30 |
| *** morganfainberg_L has quit IRC | 18:31 | |
| openstackrecheck | Console logs not available after 13:22s for check-grenade-dsvm-partial-ncpu 102568,1,856ab45 | 18:38 |
| *** afazekas_ has quit IRC | 18:41 | |
| openstackgerrit | Emily Hugenbruch proposed a change to openstack/tempest: Add tests for wait_for_server_status https://review.openstack.org/102605 | 18:47 |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full 102546,2,8a028b5 | 18:51 |
| *** harlowja has quit IRC | 18:52 | |
| *** wingwj has quit IRC | 19:00 | |
| *** wingwj has joined #openstack-qa | 19:00 | |
| *** mwagner_lap has quit IRC | 19:02 | |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for nova libvirt connection reset snapshot bug 1334398 https://review.openstack.org/102608 | 19:03 |
| *** afazekas_ has joined #openstack-qa | 19:03 | |
| *** openstackgerrit has quit IRC | 19:04 | |
| *** Longgeek has quit IRC | 19:04 | |
| openstackrecheck | Console logs not available after 13:21s for gate-cinder-pep8 102145,4,77ed7be | 19:04 |
| *** Obulpathi has quit IRC | 19:05 | |
| *** openstackgerrit has joined #openstack-qa | 19:05 | |
| *** Longgeek has joined #openstack-qa | 19:08 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Add network service tag to compute tests https://review.openstack.org/99992 | 19:08 |
| *** tpatil has joined #openstack-qa | 19:13 | |
| mgagne | is there a known transient bug with gate-tempest-dsvm-full and gate-tempest-dsvm-postgres-full? | 19:15 |
| tpatil | Can someone please review this patch https://review.openstack.org/#/c/79549/? Thanks | 19:15 |
| mgagne | Check passed 3 hours earlier but verify failed: https://review.openstack.org/#/c/91727/... | 19:15 |
| mtreinish | mgagne: http://status.openstack.org/elastic-recheck/ | 19:15 |
| mgagne | mtreinish: I've been reverifying multiple times and it fails in numerous ways | 19:16 |
| mgagne | mriedem: with different bug numbers as it failed for different reasons | 19:17 |
| openstackrecheck | Console logs not available after 13:22s for gate-nova-python26 95723,8,1aab505 | 19:18 |
| *** dustins has quit IRC | 19:18 | |
| *** Sukhdev has quit IRC | 19:18 | |
| *** dustins has joined #openstack-qa | 19:20 | |
| mtreinish | mgagne: you asked if there were known bugs, that page lists them all. If you hit a failure not caused by a bug on that list you can open a bug and submit a query so we track it. | 19:21 |
| mgagne | mtreinish: I'm well aware of this page. I rechecked/reverified against 4 different transient bugs so far but unfortunately don't have the competence to fix any of them. I was more or less checking if there was a more major issue in the order of: Yha, don't bother reverifying, it's jammed. | 19:30 |
| *** Sukhdev has joined #openstack-qa | 19:31 | |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-full 101650,2,2fed0e4 | 19:31 |
| mriedem | mgagne: bug 1320617 and bug 1334398 are active right now | 19:33 |
| mriedem | active as in hitting a lot | 19:33 |
| mriedem | so yes you'll be rechecking | 19:33 |
| mriedem | http://jogo.github.io/gate/ | 19:33 |
| *** eezhova has quit IRC | 19:34 | |
| openstackgerrit | Emily Hugenbruch proposed a change to openstack/tempest: Add tests for wait_for_server_status https://review.openstack.org/102605 | 19:35 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for nova libvirt connection reset snapshot bug 1334398 https://review.openstack.org/102608 | 19:35 |
| *** Obulpathi has joined #openstack-qa | 19:37 | |
| openstackgerrit | A change was merged to openstack-dev/devstack: Build retry loop for screen sessions https://review.openstack.org/101004 | 19:37 |
| *** openstackrecheck has quit IRC | 19:38 | |
| *** openstackrecheck has joined #openstack-qa | 19:38 | |
| *** pcrews has quit IRC | 19:39 | |
| *** pcrews_ has joined #openstack-qa | 19:39 | |
| sdague | mgagne: also part of the problem ends up being the aggregate effect | 19:42 |
| *** Obulpathi has quit IRC | 19:42 | |
| sdague | when we've got a hundred little bugs like that, the compounding chance of hitting any of them gets pretty high | 19:42 |
| mgagne | sdague: right. What's mildly infuriating is that different transitent bugs have prevented my (trivial?) change to merge for the past weeks =) | 19:43 |
| *** ajo|mac has joined #openstack-qa | 19:44 | |
| *** ajo|mac has quit IRC | 19:44 | |
| *** ajo|mac has joined #openstack-qa | 19:47 | |
| sdague | mgagne: yeh, I don't know what to say. OpenStack is a complex system. So if we test for regressions, we find race bugs in openstack itself. | 19:48 |
| sdague | so it's a balancing game between merging code, and preventing the system from being completely unusable for anyone | 19:48 |
| openstackgerrit | Matt Riedemann proposed a change to openstack/tempest: Add caller to v1 image_client wait_for_image_status timeout exception https://review.openstack.org/102617 | 19:49 |
| mriedem | mtreinish: sdague: that should help with e-r fingerprints ^ | 19:49 |
| sdague | mriedem: why? | 19:50 |
| mriedem | sdague: because this isn't good: message:"Time Limit Exceeded!" AND message:"while waiting for active, but we got queued" | 19:50 |
| mriedem | test_create_backup has been failing but it's not the only thing using that waiter | 19:50 |
| sdague | ok, so that will give you the test class that called it | 19:50 |
| sdague | but do you think they are really different bugs? | 19:50 |
| mriedem | sdague: not necessarily, but i'm not seeing other reported bugs in lp on that same failure message | 19:51 |
| mriedem | but would like to avoid a blanket 'timed out b/c of x' | 19:51 |
| *** tpatil has quit IRC | 19:51 | |
| sdague | ok, I'm fine with that | 19:51 |
| mriedem | when there are no errors in the screen logs | 19:51 |
| sdague | +2 | 19:51 |
| *** andreaf_ has joined #openstack-qa | 19:53 | |
| *** Obulpathi has joined #openstack-qa | 19:55 | |
| openstackrecheck | Console logs not available after 13:25s for check-tempest-dsvm-full 102602,1,a0c1a5c | 19:55 |
| mtreinish | mriedem: I'm more concerned that we have 2 wait_for_image_status() methods | 19:55 |
| mriedem | mtreinish: yeah, i know, that's a bigger refactor | 19:56 |
| mtreinish | sigh, actually 4... (although 2 of them use the common one) | 19:57 |
| *** sabeen1 has joined #openstack-qa | 19:58 | |
| *** sabeen1 has quit IRC | 19:58 | |
| mtreinish | mriedem: would it just be easier to refactor that method to call the one in common.waiters | 19:58 |
| *** sabeen1 has joined #openstack-qa | 19:58 | |
| mtreinish | err I guess not easily, because it's checking for slightly different exceptions | 19:59 |
| openstackgerrit | afazekas proposed a change to openstack-dev/devstack: soft-qemu handling with F20 could image https://review.openstack.org/96180 | 19:59 |
| mriedem | mtreinish: it'd take some time to make sure i didn't miss something | 19:59 |
| *** sabeen has quit IRC | 19:59 | |
| *** gszasz has quit IRC | 20:00 | |
| openstackgerrit | afazekas proposed a change to openstack-dev/devstack: soft-qemu handling with F20 could image https://review.openstack.org/96180 | 20:05 |
| *** openstackgerrit has quit IRC | 20:05 | |
| *** Longgeek has quit IRC | 20:05 | |
| *** dustins has quit IRC | 20:06 | |
| *** openstackgerrit has joined #openstack-qa | 20:06 | |
| *** Obulpathi has quit IRC | 20:08 | |
| openstackrecheck | Console logs not available after 13:22s for check-requirements-integration-dsvm 102584,2,968f4ea | 20:09 |
| *** Obulpathi has joined #openstack-qa | 20:12 | |
| *** Obulpathi has quit IRC | 20:12 | |
| *** Obulpathi has joined #openstack-qa | 20:13 | |
| mriedem | this fixes a bug that's masking other failures, when dumping console output for a server that times out waiting for state transition or goes to active: https://review.openstack.org/#/c/101362/ | 20:13 |
| mriedem | i had fixed the nova v3 client for the same issue, this fixes the v2 xml client | 20:14 |
| *** malini is now known as malini|afk | 20:14 | |
| openstackgerrit | afazekas proposed a change to openstack-dev/devstack: soft-qemu handling with F20 could image https://review.openstack.org/96180 | 20:15 |
| openstackgerrit | afazekas proposed a change to openstack-dev/devstack: soft-qemu handling with F20 could image https://review.openstack.org/96180 | 20:15 |
| *** mwagner_lap has joined #openstack-qa | 20:18 | |
| sdague | ok, is there any in flight patches on the ssh debug bits? | 20:19 |
| sdague | like something that actually adds a ping to the path? | 20:20 |
| mriedem | sdague: i'm not aware of any, mtreinish had a patch to cleanup the ping error message but mostly unrelated i think | 20:20 |
| sdague | yeh, that's actually a different path than I'm seeing die | 20:21 |
| mriedem | sdague: my most recent theories related where in ec2 testing somehow hitting/causing the leaks | 20:21 |
| sdague | mriedem: yep | 20:21 |
| mriedem | mikal added the logging but didn't dig into anything since that merged | 20:21 |
| *** afazekas_ has quit IRC | 20:21 | |
| openstackgerrit | Adam Gandelman proposed a change to openstack/tempest: Skip baremetal tests if driver not supported https://review.openstack.org/102628 | 20:21 |
| mriedem | so would be good to add some ec2 test tracing to see that network related things are cleaned up properly? | 20:21 |
| mriedem | maybe also in the ec2 api in nova | 20:22 |
| mriedem | wasn't someone saying awhile back that they had done a bunch of out of tree improvements for ec2 tests in tempest and were going to contribute those back? | 20:22 |
| openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-full 101050,3,ddffc28 | 20:22 |
| mriedem | when jogo or someone mentioned dropping support | 20:22 |
| dkranz | mtreinish: Why does pretty_tox_serial.sh look so different than pretty_tox.sh? | 20:24 |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for tempest get_console_output bug 1329563 https://review.openstack.org/102629 | 20:26 |
| mriedem | heh, bug 1334398 is now at the top of the e-r status list | 20:28 |
| openstackgerrit | Sean Dague proposed a change to openstack/tempest: ping before we ssh https://review.openstack.org/102633 | 20:32 |
| sdague | mriedem: so is 1334398 a nova bug in how we talk to libvirt, or a libvirt bug? | 20:33 |
| mriedem | sdague: well it's weird, it spiked yesterday but then falls off | 20:33 |
| mriedem | yesterday infra moved to trusty which has newer libvirt right? | 20:33 |
| mriedem | there are other libvirt bugs reported for nova with connection resets | 20:34 |
| mriedem | but they were in suspend flows | 20:34 |
| mriedem | and rescue i think | 20:34 |
| mriedem | my guess would be libvirt bug | 20:34 |
| sdague | mriedem: no, it spiked yesterday, and then we have a delay in ES data | 20:35 |
| sdague | the flatline is just the blind spot | 20:35 |
| mriedem | oh | 20:35 |
| mriedem | well then i'd guess libvirt bug since we moved to trusty | 20:35 |
| mriedem | or nova bug with newer libvirt :) | 20:35 |
| mriedem | so basically idk | 20:35 |
| sdague | right, that was my question :) | 20:35 |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-full 100950,2,c2add30 | 20:35 |
| jogo | mriedem: yeah that is correct | 20:38 |
| jogo | mriedem: actually those patches are stuck right now | 20:38 |
| jogo | let me show you | 20:38 |
| mtreinish | dkranz: because when I first wrote it there wasn't a path available in pbr to run testr in serial | 20:38 |
| mtreinish | so I had to do what pbr was doing for in the script | 20:38 |
| dkranz | mtreinish: ah, ok. Thanks. | 20:38 |
| mtreinish | it could be refactored to pass the flag to setup.py test to run it in serial | 20:39 |
| mtreinish | but I haven't seen the need to bother | 20:39 |
| jogo | mriedem: http://pastebin.com/raw.php?i=jHRe5n9f | 20:39 |
| dkranz | mtreinish: ok, np | 20:39 |
| *** andreaf_ has quit IRC | 20:39 | |
| jogo | list of patches that need to land before ec2 tempest test can land | 20:39 |
| jogo | mriedem: if you want to go through some of them | 20:40 |
| mriedem | jogo: noted, thanks - maybe we bring this in the nova meeting tomorrow | 20:41 |
| mtreinish | mriedem: yeah that's different, that fixes the log level on a retry for the remote command ping. (pinging something when ssh'd into the server) | 20:41 |
| mriedem | i.e. if we think ec2 is causing issues in the gate, and people are providing fixes and better testing, let's get attention on those | 20:42 |
| mtreinish | because right now it dumps stack traces on every retry | 20:42 |
| mriedem | jogo: i'll add to the agenda | 20:42 |
| jogo | mriedem: thanks | 20:44 |
| *** tpatil has joined #openstack-qa | 20:45 | |
| *** BobBall is now known as BobBall-Away | 20:46 | |
| mtreinish | mriedem: we don't have i8ln setup for the tempest logs. So I don't think using _() buys us anything. | 20:47 |
| mtreinish | unless I'm misunderstanding what that's used for | 20:47 |
| openstackrecheck | Console logs not available after 13:41s for check-tempest-dsvm-full 99992,2,dee239b | 20:49 |
| mriedem | mtreinish: just used to it now given the logging guidelines | 20:50 |
| mriedem | mtreinish: i wasn't -1 for that | 20:50 |
| mtreinish | although we probably should set it up for the tempest logs I guess, it's just not a high prio item | 20:51 |
| mtreinish | mriedem: oops, yeah I see that now, the first comment was -1 the second was 0 | 20:52 |
| sdague | mriedem: does your bug *always* show up in _live_snapshot? | 20:53 |
| *** ajo|mac has quit IRC | 20:53 | |
| mriedem | sdague: i moved the e-r query to _snapshot_instance | 20:54 |
| *** ajo|mac has joined #openstack-qa | 20:54 | |
| mriedem | sdague: but from the sample traces i looked at in kibana that's where it was happening | 20:54 |
| sdague | mriedem: right, but I actually think _live_snapshot is the important thing | 20:54 |
| sdague | I think it's what actually doesn't work | 20:54 |
| mriedem | i originally had it on _live_snapshot | 20:54 |
| mriedem | the hits might be the same | 20:54 |
| mriedem | sdague: right now it's 195 in 2 days either way | 20:55 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Add network service tag to compute tests https://review.openstack.org/99992 | 20:59 |
| *** ajo|mac has quit IRC | 21:02 | |
| openstackrecheck | Console logs not available after 13:23s for gate-tempest-dsvm-full 102378,1,b9d8753 | 21:02 |
| mtreinish | dims__: on https://review.openstack.org/#/c/101362/1 what is the default console length if one isn't specified? Is it the whole console output? | 21:03 |
| *** sabeen1 has quit IRC | 21:05 | |
| flwang | maurosr: piing | 21:07 |
| maurosr | flwang: hey, what's up? | 21:09 |
| flwang | maurosr: would you mind revisiting my swift patch https://review.openstack.org/99514 ? just add some comments for the case per the other review's comments, thanks | 21:10 |
| maurosr | sure | 21:10 |
| mrodden | mtreinish: had some feedback on your latest patch https://review.openstack.org/#/c/99992/3 | 21:12 |
| *** Obulpathi has quit IRC | 21:13 | |
| mtreinish | mrodden: heh, ok thanks. I'll add those files | 21:13 |
| mrodden | there was a compute/servers/test_security_groups_negative as well | 21:13 |
| mrodden | i missed in the comment | 21:13 |
| mtreinish | hopefully I won't do a s/network/negative again like I did on the first 2 revs | 21:13 |
| *** Obulpathi has joined #openstack-qa | 21:13 | |
| mrodden | lol | 21:13 |
| openstackgerrit | A change was merged to openstack/tempest: Handle backup not found in test_create_backup https://review.openstack.org/98693 | 21:14 |
| flwang | maurosr: thank you :) | 21:14 |
| openstackgerrit | A change was merged to openstack-dev/devstack: Add explicit libkrb5-dev dependency for keystone https://review.openstack.org/102600 | 21:14 |
| openstackgerrit | A change was merged to openstack-dev/devstack: Always check keystone using http https://review.openstack.org/99433 | 21:14 |
| *** ekhugen has quit IRC | 21:15 | |
| flwang | can any Tempest core bless this https://review.openstack.org/#/c/99514 ? thanks | 21:16 |
| *** sballe has joined #openstack-qa | 21:16 | |
| openstackrecheck | Console logs not available after 13:23s for check-requirements-integration-dsvm 102362,3,a542f08 | 21:16 |
| *** weshay has quit IRC | 21:17 | |
| mtreinish | mrodden: I can't find a test_security_groups_negative in compute/servers did you mean a different one? | 21:18 |
| mrodden | uhmmm | 21:18 |
| mrodden | let me check | 21:18 |
| mrodden | probably got confused | 21:18 |
| mrodden | tempest/api/compute/servers/test_server_addresses_negative.py i think it was | 21:19 |
| mtreinish | yep that's it | 21:19 |
| mrodden | k | 21:19 |
| mrodden | server_addresses and security groups are basically the same right... | 21:20 |
| mrodden | :) | 21:20 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Add network service tag to compute tests https://review.openstack.org/99992 | 21:20 |
| mtreinish | mrodden: heh, yeah I guess they are | 21:20 |
| mtreinish | mrodden: ^^^ | 21:21 |
| *** dkranz has quit IRC | 21:22 | |
| *** SridharG has quit IRC | 21:22 | |
| mrodden | yeah just looked over it quick | 21:22 |
| mrodden | thats all i could find that was missed | 21:22 |
| mrodden | +1 | 21:22 |
| *** HenryG has quit IRC | 21:23 | |
| *** SridharG has joined #openstack-qa | 21:24 | |
| mtreinish | I'm always just going to assume some are missing, it's more of a best effort thing | 21:24 |
| *** cbader has quit IRC | 21:25 | |
| *** harlowja has joined #openstack-qa | 21:28 | |
| openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-postgres-full 99370,10,9f1d838 | 21:29 |
| *** Sukhdev has quit IRC | 21:31 | |
| *** Obulpathi has quit IRC | 21:31 | |
| *** jaypipes has quit IRC | 21:34 | |
| *** SridharG has quit IRC | 21:37 | |
| *** SridharG has joined #openstack-qa | 21:38 | |
| *** SridharG has quit IRC | 21:39 | |
| openstackrecheck | Console logs not available after 13:22s for gate-cinder-pep8 101847,4,31176e4 | 21:43 |
| *** mmaglana has quit IRC | 21:50 | |
| sdague | mriedem: we don't do i18n in tempest | 21:52 |
| mriedem | sdague: alright, you can ignore the comment or respond with the same, that wasn't my -1 | 21:53 |
| sdague | the -1 was the unneed e? | 21:53 |
| mriedem | sdague: the missing log message var | 21:54 |
| sdague | oh, right | 21:54 |
| mriedem | LOG.warning('thing %s') | 21:54 |
| sdague | I deleted too much | 21:54 |
| openstackgerrit | Sean Dague proposed a change to openstack/tempest: ping before we ssh https://review.openstack.org/102633 | 21:55 |
| sdague | mriedem: thanks | 21:55 |
| openstackrecheck | Console logs not available after 13:23s for gate-ceilometer-python26 96944,3,8e219f5 | 21:56 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Make unable to ping log messages appropriate level https://review.openstack.org/95815 | 21:56 |
| *** ajo|mac has joined #openstack-qa | 21:59 | |
| *** dims__ has quit IRC | 22:01 | |
| mtreinish | sdague: do we allow ping in our secgroup for testing? I was about to +2 your patch, but I remembered we could be blocking ping | 22:02 |
| *** nikhil___ is now known as nikhil___|afk | 22:04 | |
| mtreinish | I guess when jenkins comes back that will give the answer | 22:07 |
| sdague | I'm pretty sure we turn on icmp | 22:07 |
| *** markmcclain has quit IRC | 22:08 | |
| mtreinish | ok, that's not surprising I guess | 22:09 |
| mtreinish | sdague: oh, looks like you broke the ssh unit tests... | 22:09 |
| openstackrecheck | Console logs not available after 13:24s for gate-python-glanceclient-python26 82126,15,a6fe6da | 22:09 |
| mtreinish | sdague: http://logs.openstack.org/33/102633/2/check/gate-tempest-python27/f987161/console.html | 22:10 |
| sdague | huh, I thought I ran that locally | 22:10 |
| sdague | seriously... default tox doesn't run py27? | 22:11 |
| * sdague scowls at mtreinish | 22:11 | |
| openstackgerrit | Sean Dague proposed a change to openstack/tempest: ping before we ssh https://review.openstack.org/102633 | 22:12 |
| openstackgerrit | Sean Dague proposed a change to openstack/tempest: add py27 to default tox https://review.openstack.org/102661 | 22:12 |
| mtreinish | sdague: oops, I'll fix that right now | 22:12 |
| sdague | mtreinish: just did | 22:12 |
| mtreinish | ok you win | 22:12 |
| sdague | also, for someone that says we don't support 33... you have unit testing rules for it :) | 22:13 |
| mtreinish | heh, yeah I know. We also have py26 | 22:13 |
| mtreinish | but I know that python3x fails tox when it setups the venv because of boto | 22:14 |
| mtreinish | I can't remember if there is another import issue | 22:14 |
| mtreinish | I'm going to bump that to 34 eventually | 22:14 |
| *** gszasz has joined #openstack-qa | 22:15 | |
| mriedem | mtreinish: i thought py26 was dropped? | 22:15 |
| mtreinish | it's not supported, and we don't test it. But I think it works | 22:16 |
| mtreinish | I never got around to removing the tox py26 unit test job | 22:16 |
| mriedem | mtreinish: yeah it works, we still never got the testr discovery patch thingy going | 22:16 |
| mriedem | so we exclude all of the testscenarios negative tests stuff | 22:16 |
| mriedem | moving to rhel 7 for juno though so we should be using py27 then | 22:17 |
| mriedem | \o/ | 22:17 |
| mriedem | i shouldn't say 'we' since i'm not on that team anymore | 22:17 |
| *** sabeen1 has joined #openstack-qa | 22:17 | |
| mtreinish | heh, they'll keep pulling you back in :) | 22:17 |
| mriedem | it's true | 22:23 |
| openstackrecheck | Console logs not available after 13:22s for gate-tempest-dsvm-neutron-large-ops 99340,11,253fe7f | 22:23 |
| *** dims__ has joined #openstack-qa | 22:25 | |
| dims__ | mtreinish, yes, default is all the console output | 22:25 |
| mtreinish | dims__: ok cool, then I'm +2 :) | 22:26 |
| mtreinish | for whatever reason I thought the default was like 25 or something | 22:27 |
| openstackgerrit | Sean Dague proposed a change to openstack/tempest: ping before we ssh https://review.openstack.org/102633 | 22:32 |
| sdague | mtreinish: so that still doesn't pass unit tests | 22:32 |
| sdague | but I've run out of time tonight to figure out the last break | 22:33 |
| mtreinish | sdague: hmm, ok I'll take a look later | 22:33 |
| mtreinish | but let's assign blame to jaypipes, he wrote all of that :) | 22:33 |
| dims__ | lol +1 | 22:33 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for neutron bug 1334109 https://review.openstack.org/102558 | 22:34 |
| sdague | well mostly, it throws an exception | 22:34 |
| sdague | but I don't get how it didn't before | 22:34 |
| *** openstackgerrit has quit IRC | 22:35 | |
| mtreinish | sdague: heh, it passed for me locally... | 22:36 |
| mtreinish | oops nm, ran tox on the wrong dir | 22:36 |
| *** openstackgerrit has joined #openstack-qa | 22:36 | |
| openstackrecheck | Console logs not available after 13:26s for gate-python-cinderclient-python33 102586,6,ce44d12 | 22:36 |
| *** openstackrecheck has quit IRC | 22:44 | |
| *** openstackrecheck has joined #openstack-qa | 22:44 | |
| *** markmcclain has joined #openstack-qa | 22:45 | |
| *** aimon_ has quit IRC | 22:47 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for tempest get_console_output bug 1329563 https://review.openstack.org/102629 | 22:48 |
| *** markmcclain has quit IRC | 22:49 | |
| mtreinish | sdague: I think it was just the extra call path was adding ~1 sec to the execution which was exceeding the timeout set in the client | 22:49 |
| *** sabeen1 has quit IRC | 22:50 | |
| mtreinish | it passed when I bumped the timeout, of course I had to change one of the time checks we were using to verify it attempted the correct number of times | 22:50 |
| mtreinish | I'll push out a rev with the changes | 22:51 |
| openstackgerrit | A change was merged to openstack/tempest: Add "create a server" test with server group https://review.openstack.org/86792 | 22:51 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: ping before we ssh https://review.openstack.org/102633 | 22:52 |
| mtreinish | sdague: ^^^ | 22:52 |
| *** openstackrecheck has quit IRC | 22:52 | |
| *** openstackrecheck has joined #openstack-qa | 22:52 | |
| *** mriedem has quit IRC | 22:54 | |
| *** mdenny has quit IRC | 22:59 | |
| *** pcrews_ has quit IRC | 23:00 | |
| *** pcrews has joined #openstack-qa | 23:00 | |
| *** nsaje has quit IRC | 23:01 | |
| *** markmcclain has joined #openstack-qa | 23:01 | |
| *** nsaje has joined #openstack-qa | 23:05 | |
| openstackrecheck | Console logs not available after 13:27s for check-tempest-dsvm-postgres-full 102660,1,bba4f83 | 23:06 |
| *** rbak has quit IRC | 23:07 | |
| *** Sukhdev has joined #openstack-qa | 23:14 | |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-full 102653,3,ff70f74 | 23:19 |
| *** rodrigods_ has joined #openstack-qa | 23:21 | |
| *** atiwari has quit IRC | 23:27 | |
| *** rbak has joined #openstack-qa | 23:28 | |
| *** aimon_ has joined #openstack-qa | 23:29 | |
| *** oomichi has joined #openstack-qa | 23:30 | |
| openstackrecheck | Console logs not available after 13:23s for gate-swift-dsvm-functional 99276,3,89d7dc5 | 23:33 |
| *** sballe has quit IRC | 23:38 | |
| *** GMann has joined #openstack-qa | 23:46 | |
| openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-full 102595,1,923a299 | 23:46 |
| *** hemna is now known as hemna_ | 23:49 | |
| *** ajo|mac has quit IRC | 23:52 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!