Wednesday, 2014-10-29

openstackgerritJoe Gordon proposed a change to openstack-dev/grenade: Add comment explaining where javelin logs are found  https://review.openstack.org/11958700:04
openstackgerritAdolfo Duarte proposed a change to openstack/tempest: Test Suite for DVR (Distributed Virtual Routing) scenarios  https://review.openstack.org/13160600:06
*** achanda has quit IRC00:15
*** achanda has joined #openstack-qa00:15
*** dimtruck is now known as zz_dimtruck00:16
*** zz_dimtruck is now known as dimtruck00:16
*** achanda has quit IRC00:19
*** rushiagr is now known as rushiagr_away00:20
*** dimtruck is now known as zz_dimtruck00:25
*** dmorita has joined #openstack-qa00:29
*** patrickeast has quit IRC00:36
*** bknudson has joined #openstack-qa00:38
*** Longgeek has joined #openstack-qa00:47
*** Longgeek has quit IRC00:48
*** Longgeek has joined #openstack-qa00:48
*** Yogi has quit IRC00:51
*** JayJ has joined #openstack-qa00:53
*** bknudson has left #openstack-qa00:56
*** e0ne has quit IRC00:59
*** catohornet has quit IRC01:01
*** yamahata has joined #openstack-qa01:03
*** Yogi has joined #openstack-qa01:10
*** e0ne has joined #openstack-qa01:13
*** Yogi has quit IRC01:19
*** Longgeek has quit IRC01:19
*** yamamoto_ has joined #openstack-qa01:20
*** Yogi has joined #openstack-qa01:22
*** yamahata has quit IRC01:22
*** yamahata has joined #openstack-qa01:22
*** marun has quit IRC01:29
*** e0ne has quit IRC01:33
*** catohornet has joined #openstack-qa01:35
*** rbak has joined #openstack-qa01:42
*** weshay has quit IRC01:49
*** amotoki has joined #openstack-qa01:59
*** nosnos has joined #openstack-qa02:06
*** dims__ has quit IRC02:14
*** dims__ has joined #openstack-qa02:15
*** dims__ has quit IRC02:17
*** morganfainberg is now known as only_zuul02:18
*** only_zuul is now known as morganfainberg02:18
openstackgerritA change was merged to openstack/tempest: Fix and re-enable test_disassociate_not_associated_floating_ip  https://review.openstack.org/12432302:20
openstackgerritYAMAMOTO Takashi proposed a change to openstack-dev/devstack: Stop setting deprecated OVS.enable_tunneling option  https://review.openstack.org/13132702:27
*** Poornima has joined #openstack-qa02:31
*** mriedem has quit IRC02:40
*** dims__ has joined #openstack-qa02:47
*** dims__ has quit IRC02:52
*** chandankumar has joined #openstack-qa02:57
*** chandankumar has quit IRC02:58
*** JayJ has quit IRC03:12
*** Yogi has quit IRC03:17
*** yamahata has quit IRC03:22
*** vikas has joined #openstack-qa03:26
*** vikas has quit IRC03:27
openstackrecheckConsole logs not available after 13:27s for check-tempest-dsvm-postgres-full-juno 131364,1,daa1db903:27
*** rbak has quit IRC03:28
*** rbak has joined #openstack-qa03:29
*** nosnos has quit IRC03:46
*** nosnos has joined #openstack-qa03:46
*** nosnos has quit IRC03:51
*** catohornet has left #openstack-qa03:56
*** yamahata has joined #openstack-qa04:05
*** chandankumar has joined #openstack-qa04:05
*** harlowja is now known as harlowja_away04:21
*** nosnos has joined #openstack-qa04:27
*** chandankumar has quit IRC04:32
*** Poornima has quit IRC04:34
*** pcrews has quit IRC04:37
openstackgerritEli Qiao proposed a change to openstack/tempest: Add BadRequest exception in test_floating_ips_actions case  https://review.openstack.org/13136404:39
*** yfried has joined #openstack-qa04:40
*** pcrews has joined #openstack-qa04:41
openstackrecheckopenstack/nova change: https://review.openstack.org/122633 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/137351304:45
*** achanda has joined #openstack-qa04:49
*** morganfainberg is now known as mightbestevemar04:55
*** pcrews has quit IRC04:57
*** mightbestevemar is now known as morganfainberg05:04
*** Poornima has joined #openstack-qa05:05
openstackgerritKen'ichi Ohmichi proposed a change to openstack/tempest: Fix the flavor ram string value  https://review.openstack.org/13163705:17
*** yfried has quit IRC05:25
*** pcrews has joined #openstack-qa05:26
*** rushiagr_away is now known as rushiagr05:29
*** jecarey_ has quit IRC05:32
*** pcrews has quit IRC05:32
openstackgerritKen'ichi Ohmichi proposed a change to openstack/tempest: Do not merge: Test v2.1 API on the gate  https://review.openstack.org/13163905:34
openstackgerritClark Boylan proposed a change to openstack-infra/devstack-gate: Enable nova multi host on aiopcpu tests  https://review.openstack.org/13160405:36
openstackrecheckopenstack/horizon change: https://review.openstack.org/129379 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/137351305:42
*** chandankumar has joined #openstack-qa05:44
openstackrecheckopenstack/horizon change: https://review.openstack.org/121935 failed gate-horizon-pep8 in the gate queue with an unrecognized error05:46
*** Madan has joined #openstack-qa05:56
openstackgerritClark Boylan proposed a change to openstack-infra/devstack-gate: Enable nova multi host on aiopcpu tests  https://review.openstack.org/13160406:06
*** afazekas is now known as __afazekas06:11
*** achanda has quit IRC06:12
*** achanda has joined #openstack-qa06:14
*** achanda has quit IRC06:20
*** achanda has joined #openstack-qa06:25
openstackrecheckopenstack/nova change: https://review.openstack.org/121965 failed gate-tempest-dsvm-full in the gate queue with an unrecognized error06:27
*** yfried has joined #openstack-qa06:37
openstackgerritAbhishek G M proposed a change to openstack/tempest: Test to verify  shared attribute of network  https://review.openstack.org/12499806:37
*** achanda has quit IRC06:39
yfriedmasayukig: ping06:53
yfriedgmann: ping06:54
gmannyfried: pong06:54
yfriedgmann: remember this https://review.openstack.org/#/c/120070/ ?06:54
gmannyfried: ah yes06:55
yfriedgmann: so I think this is finally ready. the bug is fixed and the test passed on my machine with 100 vms06:56
yfriedgmann: though this requires some tweaks ad06:56
yfriedat the gate06:56
*** e0ne has joined #openstack-qa06:56
yfriedgmann: ie - build timeout should be extended an some quotas need to be disabled06:57
yfriedgmann: but I wonder if the existing 100 machines isn't too much for a gate-vm06:57
yfriedgmann: run-time on my machine is ~700sec per iteration, with 3 iterations so its ~30min test06:58
*** evgenyf has joined #openstack-qa06:58
yfriedgmann: and I'm using a bare-metal server instead of a VM with questionable resources06:59
yfriedgmann: I wonder if we really need to test 100 vms in the gate. I also wonder why it has to run 3 times06:59
gmannyfried: 3 times is fine as test timeout is 500 sec for single tests07:00
gmannyfried: but as you mentioned it takes ~700 sec per iteration, can be issue on gate07:00
gmannyfried: bug fixed you mean - bug#1372049?07:01
yfriedgmann: I'm also concerned that 100 vms is overtaxing the single-node setup. not sure that openstack is designed to support that07:01
yfriedgmann: yep. that's the bug07:02
yfriedgmann: fixed on the nova side07:02
yfriedgmann: I think 50 machines is "large-scale" enough07:03
*** afazekas has joined #openstack-qa07:03
yfriedgmann: brb07:03
gmannyfried: humm actually I am not in favor of 50 as we should not truncate actual large ops testing07:04
gmannyfried: but  Nova has fixed the max thread issue in master means for Kilo. are they planning to backport those for J and I also?07:06
gmannyfried: bcz for us(tempest), without backport it would not work.07:06
gmannyfried: another idea i have that let large ops build 100 VM and you can perform connectivity check for half of them?07:07
*** cmyster has joined #openstack-qa07:11
*** cmyster has quit IRC07:11
*** cmyster has joined #openstack-qa07:11
*** cmyster has quit IRC07:15
*** SridharG has joined #openstack-qa07:17
*** yfried is now known as yfried|afk07:19
*** yfried|afk is now known as yfried07:19
yfriedgmann: well, I don't remember if we've discussed this or not, but the current test doesn't do ANYTHING07:21
yfriedgmann: it checks that nova can process a request for multiple vms. it could easily be part of the nova tree07:22
yfriedgmann: I think we can drop that for a meaningful integration test07:23
yfriedgmann: we aren't loosing anything by reducing the number of VMs because we didn't have anything in the first place. I think I'll try and bring it up at tomorrow's mtg.07:24
*** nosnos has quit IRC07:25
yfriedgmann: or, the "scenario" could move to nova api because it's not exercising any other component ATM07:26
*** nosnos has joined #openstack-qa07:27
yfriedgmann: I'll ask about backporting the patch to K or I07:27
gmannyfried: yes, that will be good to discuss in tomorrow meeting.07:28
*** SridharGaddam has joined #openstack-qa07:29
yfriedgmann: assuming there is one?07:29
gmannyfried: IMO, we should keep it in scenario and can extend this tests for other component too may be by reducing number of VMs or by having other operation (like connectivity) on limited VMs07:30
yfriedgmann: if we have network in the tenant, it's a very bad hack to get only 50% of the VMs to connect to it07:31
yfriedgmann: easier to lower the total num07:32
gmannyfried: ahh. if there is no meeting, you can discuss in summit on last day session which is shared with infra team07:32
gmannyfried: hoping you will be there :)07:33
yfriedgmann: too bad I'm leaving Wednesday night...07:33
gmannyfried: even I am also not able to make :(07:35
*** cmyster has joined #openstack-qa07:36
yfriedgmann: there was a pad with summit preparations. do you know where it is?07:36
gmannyfried: https://etherpad.openstack.org/p/kilo-qa-summit-topics07:36
yfriedgmann: tnx07:36
gmannyfried: np!07:37
yfriedgmann: does "interested" means "would like to attend" or "would like to speak"?07:38
gmannyfried: "would like to attend"07:39
yfriedgmann: oh. then I'm interested in all of them... :)07:39
gmannyfried: :)07:39
*** yamamoto_ has quit IRC07:42
yfriedgmann: you wouldn't happen to know of a similar pad for network/neutron, would you?07:42
gmannyfried: sorry, no idea07:43
*** cmyster has quit IRC07:46
*** cmyster has joined #openstack-qa07:47
*** cmyster has quit IRC07:47
*** cmyster has joined #openstack-qa07:47
*** sergsh has joined #openstack-qa08:02
*** gfidente has joined #openstack-qa08:02
*** qba73 has joined #openstack-qa08:03
*** jlibosva has joined #openstack-qa08:12
openstackgerritKen'ichi Ohmichi proposed a change to openstack/tempest: Add internal methods for setting compute clients  https://review.openstack.org/13166408:14
openstackgerritKen'ichi Ohmichi proposed a change to openstack/tempest: Use JSON clients if disabling XML compute support  https://review.openstack.org/13166508:14
*** e0ne has quit IRC08:19
*** yamahata has quit IRC08:22
*** rushiagr is now known as rushiagr_away08:27
*** k4n0 has joined #openstack-qa08:28
*** tkammer_ has joined #openstack-qa08:32
*** cmyster has quit IRC08:34
*** tkammer has quit IRC08:35
*** markus_z has joined #openstack-qa08:37
*** neeti has joined #openstack-qa08:37
openstackrecheckopenstack/horizon change: https://review.openstack.org/131645 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/137351308:38
yfriedafazekas: is there a QA mtg this week? wiki says "No meeting next week" but I assume it refers to the summit week08:38
*** cmyster has joined #openstack-qa08:39
*** rushiagr_away is now known as rushiagr08:39
*** jordanP has joined #openstack-qa08:44
*** X019 has joined #openstack-qa08:46
*** e0ne has joined #openstack-qa08:48
*** jpich has joined #openstack-qa08:50
*** safchain has joined #openstack-qa08:50
*** nikesh_vedams_ has joined #openstack-qa08:51
openstackgerritDougal Matthews proposed a change to openstack/tempest: Correct docstring indentation  https://review.openstack.org/13136608:52
*** X019 has quit IRC08:52
nikesh_vedams_i am trying to deploy devstack on RHEL-6.508:53
nikesh_vedams_but i am getting below error08:53
nikesh_vedams_ http://paste.openstack.org/show/126342/08:53
nikesh_vedams_did any-one face this issue08:53
nikesh_vedams_?08:54
*** gszasz has joined #openstack-qa08:54
*** e0ne has quit IRC08:56
*** tkammer_ has quit IRC08:57
*** cmyster has quit IRC09:01
*** cmyster has joined #openstack-qa09:01
*** qba73 has quit IRC09:05
*** aix has joined #openstack-qa09:11
*** tkammer_ has joined #openstack-qa09:12
*** dims__ has joined #openstack-qa09:15
*** dims__ has quit IRC09:19
yfriednikesh_vedams_: are you pulling latest master (Kilo)?09:29
*** cmyster has quit IRC09:30
*** zz_johnthetubagu is now known as johnthetubaguy09:30
nikesh_vedams_yfried: yes09:34
nikesh_vedams_but after that i changed to juno09:34
nikesh_vedams_but still i am getting error09:34
*** cmyster has joined #openstack-qa09:35
*** cmyster has quit IRC09:35
*** cmyster has joined #openstack-qa09:35
yfriednikesh_vedams_: don't know if that's your problem, but RHEL6 is using python26 I don't think juno is supported on el6.09:36
*** ajo_ has joined #openstack-qa09:39
nikesh_vedams_yfried: i think rhel6 supports devstack09:40
*** achanda has joined #openstack-qa09:40
*** afazekas_beer is now known as psedlak09:40
nikesh_vedams_yfried: yfried09:41
nikesh_vedams_yfried:http://docs.openstack.org/developer/devstack/09:41
yfriednikesh_vedams_: devstack yes, but not juno.09:41
nikesh_vedams_kkk i  am installing devstack with juno on rhel609:42
nikesh_vedams_kkk09:42
yfriednikesh_vedams_: I know RedHat aren't doing RHEL-OSP-6 on el6. only for el709:42
nikesh_vedams_so icehouse will work09:42
nikesh_vedams_am i right09:42
nikesh_vedams_?09:42
yfriednikesh_vedams_: hopefully09:42
yfriednikesh_vedams_: let me know how it goes09:42
yfriednikesh_vedams_: are you intersted in devstack only? you could go for RDO09:43
nikesh_vedams_kkk i am doing now git checkout stable/icehouse09:43
nikesh_vedams_what is RDO?09:44
yfriednikesh_vedams_: https://openstack.redhat.com/Main_Page09:44
*** achanda has quit IRC09:44
yfriednikesh_vedams_: btw, what's "kkk"?09:45
nikesh_vedams_oh sorry09:46
nikesh_vedams_i mean okk09:46
yfriednikesh_vedams_: oh. well good luck09:46
*** cmyster has quit IRC09:58
openstackrecheckopenstack/nova change: https://review.openstack.org/129276 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/137351310:00
*** bkopilov has quit IRC10:00
*** bkopilov has joined #openstack-qa10:00
*** ylobankov has joined #openstack-qa10:02
*** cmyster has joined #openstack-qa10:06
*** yfried is now known as yfried|afk10:07
*** marun has joined #openstack-qa10:09
*** yamahata has joined #openstack-qa10:10
*** yfried|afk is now known as yfried10:12
*** jecarey has joined #openstack-qa10:12
*** e0ne has joined #openstack-qa10:18
*** malini has joined #openstack-qa10:18
*** malini has quit IRC10:18
*** malini has joined #openstack-qa10:19
openstackgerritJohn Davidge proposed a change to openstack-dev/devstack: Add IPv6 support for tenant data network  https://review.openstack.org/8798710:21
*** aix has quit IRC10:21
*** cmyster has quit IRC10:24
*** cmyster has joined #openstack-qa10:25
*** amotoki has quit IRC10:27
*** mpaolino has joined #openstack-qa10:28
*** dims__ has joined #openstack-qa10:30
openstackgerritchandan kumar proposed a change to openstack/tempest: Fixes volume client in volume tests  https://review.openstack.org/13169310:33
chandankumargmann, jordanP Hello10:33
chandankumargmann, jordanP please have a look at this patch: https://review.openstack.org/13169310:34
jordanPchandankumar, i'd rather wait for jenkins :)10:34
chandankumarjordanP, :)10:35
*** yfried is now known as yfried|afk10:38
*** rushiagr is now known as rushiagr_away10:39
*** evgenyf has quit IRC10:44
openstackgerritFlavio Percoco proposed a change to openstack-infra/devstack-gate: Set ZAQAR_BACKEND in localrc if present  https://review.openstack.org/12822410:50
*** cmyster has quit IRC10:52
*** aix has joined #openstack-qa10:53
*** cdent has joined #openstack-qa10:56
*** evgenyf has joined #openstack-qa11:05
sdagueianw: so I don't actually know how to do the bashate release because mrodden worked it out and was doing it. If you figure out instructions, I'm happy to cut it (or figure out how to give you permissions to do it)11:06
ianwsdague: ok, i can confer with him.  he said something about the PTL doing it11:07
sdagueyeh, he's the bashate ptl11:07
sdaguebut I'm sure we could open up the permissions to give core release authority11:07
*** marun has quit IRC11:10
*** boris-42 has quit IRC11:11
openstackgerritIan Wienand proposed a change to openstack-dev/devstack: Revert "Single quote iniset argument in merge_config_file"  https://review.openstack.org/13133411:12
*** marun has joined #openstack-qa11:13
*** yamahata has quit IRC11:17
openstackgerritChris Dent proposed a change to openstack/tempest: Add scenario test for swift-proxy ceilometer middleware  https://review.openstack.org/11597111:19
vrovachevsdague: ping11:19
openstackgerritChris Dent proposed a change to openstack-dev/devstack: Fix up file-matching in bashate tox test  https://review.openstack.org/12880911:21
*** tosky has joined #openstack-qa11:24
*** aysyd has joined #openstack-qa11:25
*** rushiagr_away is now known as rushiagr11:25
sdaguevrovachev: don't just ping, if you have a question ask it. :P11:27
sdagueianw: so... why was E021 added to bashate anyway?11:27
sdaguebecause we know it's bash, not posix shell11:28
jordanPgmann, hi, I don't get the comment on : https://review.openstack.org/#/c/116452/2/tempest/services/object_storage/account_client.py can you explain plz ?11:28
cdentianw: I trod on your stuff again, I assumed you were sleeping (128809 above).11:29
*** dmorita has quit IRC11:29
vrovachevsdague: http://th04.deviantart.net/fs70/PRE/f/2011/288/c/2/okay_by_rober_raik-d4cw242.png  :) About patch https://review.openstack.org/#/c/67164/ . After amerge patch https://review.openstack.org/#/c/114514/ ceilo works faster, and tests tests don't fails11:31
cdentmeh, the provided advice did not fix it11:31
sdaguevrovachev: but it's still an unbounded race condition right? Like there is no way to 100% ensure that it runs in the window11:32
sdaguewe really can't add new tests that have known non deterministic behavior11:32
*** andreaf_ has joined #openstack-qa11:33
cdentsdague: that's going to be case with anything driven by notifications, not just ceilometer related tests11:34
cdentI agree that non-deterministic behavior needs to be shot dead11:34
cdentbut when so much of the system is, that's problematic...11:35
sdaguecdent: right, this is why there was a bunch of discussion about getting flush APIs in the right places11:35
cdentIf we want to get reid of non-determinism, then we should probably not boot instances, because who knows how long that's going to take...11:36
*** andreaf_ is now known as andreaf11:36
cdent;)11:36
sdague:P11:36
openstackgerritA change was merged to openstack/tempest: Add list-networks test for os-networks API  https://review.openstack.org/11294411:36
cdentwhere is this "flush" discussion happening? Can you point me at some?11:36
sdagueit's been kind of spread out over IRC, gerrit reviews, I don't remember if it was on the list or not11:37
*** Poornima has quit IRC11:37
DinaBelovasdague, well, cdent is right.... I'm kind of curious what to do with these tests :) we *need* them to define if ceilometer is working in (let's say in 99% of the cases when testing lab is behaving ok itself), but that  needs us to rely on the chain something-queue-something is working11:38
sdagueso I remain -2 on adding new ceilometer notification test to tempest, because we've actually had to skip every one we've added for high failure rates, which really to me means we need a different approach11:38
vrovachevsdague: but we sure, if test falls,  it will not be guilty ceilometer11:38
sdagueDinaBelova: these seem like the sort of thing to pull back into ceilometer functional test, where you could make a white box internal call to trigger notifications flushing11:38
sdaguevrovachev: that's not been true in the past11:39
*** beagles is now known as beagles_brb11:39
sdaguebecause this class of tests has been *so* problematic11:39
vrovachevsdague:  then ceilo was different :)11:39
cdentFor the sake of fixing my ignorance can someone define the mechanism for "notifications flushing"? How does that work?11:40
sdaguecdent: so the issue now is the fact that a bunch of what happens only happens when periodic jobs trigger11:40
sdaguewhich in a real environment... is some random time in the future that you don't know11:40
cdentI think we may have some language difficulties going on here. When a ceilometer person is talking about a "notification" they usually mean something that some other service has put on the message bus: that service is in charge of when that happens.11:42
cdentThose notifications are consumed as meters.11:42
cdentThere are also pollings, which happen on a period11:42
cdentceilometer is in charge of that period11:43
sdagueyep11:43
cdentso I'm a bit confused11:43
cdentwhere is the "flush"?11:43
cdentthat is: what is responsible for it?11:43
sdaguethe flush would have to be able to trigger the periodic jobs of the servers on demand11:43
sdaguewhich, yes, is bigger than ceilometer11:43
sdaguebut is needed for ceilometer verification in a deterministic way11:44
sdagueat the integration level11:44
cdentso you're imagingin that a service which does notifications could be inspired to dump its notificaitons _now_ based on recieving a signal of some kind11:44
sdagueyep11:44
cdentSIGUSR2 to process whateves11:44
sdagueor a network API call11:44
sdague /admin/flush11:44
cdentnetwork api call not very deterministic on a non-deterministic network ;)11:45
sdagueit is if it's synchronous11:45
sdagueit would be a blocking call11:45
cdentWhat I mean is that sometimes the network simply isn't behaving as expected.11:45
cdentso you can't even get to the API11:45
cdentit was mostly a joke11:46
sdague:)11:46
cdentbut is an effort to point out: everything is really fragile11:46
sdagueanyway, there is a lot of history about the unreliability of these tests in the past, so we should really have an overview document about the test plan and approach here and why it's not going to add any new gate failures11:46
cdentDo you think the idea of getting services to be flushable has traction11:46
*** X019 has joined #openstack-qa11:47
sdaguehonestly, I don't know, but it's been something I brought up at least 6 months ago, and no one has investigated11:47
*** SridharG1 has joined #openstack-qa11:48
cdent:(11:48
* cdent needs clones11:48
sdaguebut it's *really* important that we don't add anything to tempest that has a known non trivial race chance. And because these tests have had giant races in the past, the burden of proof is back on the ceilometer team to prove that.11:48
*** mwagner_lap has quit IRC11:49
sdaguebecause I've lost too many days of work figuring out that the ceilometer tests were broken by design and need to be turned off. :(11:49
*** SridharG has quit IRC11:49
ianwcdent: ok, just a rebase?11:49
cdentianw: yeah, but it didn't fix it11:49
cdentsdague: I feel your pain. I too see a lot of broken by design.11:50
sdagueyep, we've got spades of it all over11:50
sdaguelets try to fix that :)11:50
ianwcdent: that change can't merge until a new release of bashate11:50
cdentianw: yeah, I realized that after the fact11:50
cdent(and seeing you and sdague talking earlier about releasing)11:51
*** jhenner has joined #openstack-qa11:51
cdentsdague: yes, onward etc11:51
openstackgerritAbhishek G M proposed a change to openstack/tempest: Test to verify  shared attribute of network  https://review.openstack.org/12499811:53
openstackgerritMonty Taylor proposed a change to openstack-infra/devstack-gate: Remove nosetests.xml reference  https://review.openstack.org/13171411:53
*** Poornima has joined #openstack-qa11:54
*** X019 has quit IRC11:58
*** yfried|afk is now known as yfried11:59
*** X019 has joined #openstack-qa12:00
*** SridharG1 has quit IRC12:09
*** weshay has joined #openstack-qa12:11
*** nosnos has quit IRC12:15
*** Yogi has joined #openstack-qa12:18
*** qba73 has joined #openstack-qa12:18
*** jecarey has quit IRC12:20
*** JayJ has joined #openstack-qa12:20
*** Madan has quit IRC12:21
*** eglynn has joined #openstack-qa12:22
*** dims__ has quit IRC12:24
*** dims__ has joined #openstack-qa12:24
*** JayJ has quit IRC12:25
*** Yogi has quit IRC12:26
*** JayJ has joined #openstack-qa12:26
*** SridharG has joined #openstack-qa12:30
*** JayJ has quit IRC12:30
eglynncdent, sdague: just catching up with the backlog on the notification "flushing" notion12:31
eglynn... surely we're not in general talking about *periodic* notifications here?12:31
eglynn(... e.g. the nova instance audit)12:31
eglynn... rather we're talking about resource-lifecycle notification driven by the actual state transitions?12:31
yfriednikesh_vedams_: did you solve the problems?12:31
eglynnI update an image in glance => I expect an image.updated notification12:31
sdagueeglynn: my understanding on why this stuff failed in the past was not all resource lifecycle events happen immediately12:32
eglynnsdague: that's interesting, because I though the emission of such notifications was in-line in the nova workflow initiated by the API call12:33
eglynnsdague: ... so adding a new flush API wouldn't necessarily help, since the notifications are API-triggered already12:33
sdaguebut, again, I think the point is we need test plan overview here that ensures that's not the case. If we are sure we're not adding more non determinism into the mix I'm fine with moving forward, but I don't feel like that's clear12:33
eglynnsdague: OK, so if I hearing correctly, a test plan for what we intend to keep in tempest and what we'll move to the new in-tree functional test suite in kilo?12:34
eglynnsdague: ... yes, agreed, that could be an output from the related summit discussions12:35
openstackgerritA change was merged to openstack-infra/devstack-gate: yum non interactive  https://review.openstack.org/13071912:35
sdagueeglynn: well, more specifically, for stuff coming into tempest, any possible race conditions that exist and why they are not likely to show up in test runs.12:35
sdaguethat's the piece which is missing really12:35
sdagueand what we need to evaluate adding new tests, because those tests depend on a lot of behind the scenes things happening, and a clear analysis of the expected interaction points behind the scenes, where any of them might fail (or how they can't fail) and what that means for the likelihood that the test ends up non-deterministic12:36
eglynnsdague: so just to clarify, by "race" here we mean a test assertion that depends on an synchronous event, and how we can reasonably and reliability bound the wait for that event to occur?12:37
eglynnsdague: if so, agreed, that analysis of the moving parts and their interactions should be written down12:38
sdagueeglynn: yes, if you mean "asynchronous"12:38
eglynnsdague: haha, yep, Fruedian slip on my part :)12:38
sdague:)12:38
sdagueyeh, I think if we had those interactions written down, and the expecations, I'd feel a lot more comfortable here12:39
sdaguebecause in the past we did a lot of whack-a-mole, and I'd like to avoid that going forward12:39
eglynnsdague: yep, I think we can commit to doing that for the ceilometer tests that will remain in, or be added to, tempest12:39
sdagueawesome12:39
sdaguethanks12:39
eglynnsdague: understood, that'll be something I'll aim for immediately post-summit (once the in-tree test approach is clearer)12:40
sdaguesounds perfect, thank you12:40
openstackgerritBenny Kopilov proposed a change to openstack/tempest: compute-image : create server from saved snapshot image  https://review.openstack.org/12089912:40
*** Krast has joined #openstack-qa12:48
*** jecarey has joined #openstack-qa12:49
nikesh_vedams_yfried:12:51
nikesh_vedams_i am trying12:51
nikesh_vedams_i will let you know12:51
*** dkranz has joined #openstack-qa12:53
cdentIt occurs to me, sdague, that many of the ceilometer+notifications problems that were happening in the past were probably because this hadn't been fixed: I1c06d0511fbf93050cda56d9d2de0ff00813dfb612:54
*** X019 has quit IRC12:54
cdentoops, sorry: https://review.openstack.org/#/c/113522/12:54
sdaguecdent: so, that's possible. But I still think that given history, we should make sure we're not introducing similar kinds of issues. Instead of just hoping that we aren't.12:55
cdentYes, I agree, was just adding a bit more data to the shared mind.12:55
sdagueyep12:55
DinaBelovacdent, thanks for the lnk12:55
*** tosky has quit IRC12:57
eglynncdent: I don't think that was the problem in the problem nova notification test12:57
cdentthe other main factor was that the collectors were too slow12:57
eglynncdent: ... IIUC that test didn't follow closely after a service/rabbitmq restart?12:57
cdentI don't think there's any one single cause, its a complex of things, and we have to recognize that each change will push us along12:58
cdentThat's the trouble with what we're (inaccurately) calling race conditions: they aren't single source.12:59
eglynncdent: sure, there may be multiple causes, nova tardiness in emitting the notification, ceilo slowness in ingesting it etc13:00
eglynncdent: . ... I just don't think amqp restart was a factor in that test failure13:00
cdentOkay, but I didn't say it was.13:01
cdent" many of the ceilometer+notifications problems"13:02
*** markus_z has quit IRC13:03
*** malini has quit IRC13:04
eglynncdent: apologies, I assumed relevance to the prior conversation on notification test failures in tempest13:04
*** Yogi has joined #openstack-qa13:05
*** yfried is now known as yfried|afk13:05
*** mestery has quit IRC13:08
*** yfried|afk is now known as yfried13:08
*** yamahata has joined #openstack-qa13:08
*** amotoki has joined #openstack-qa13:09
*** markmcclain has joined #openstack-qa13:09
*** marun_ has joined #openstack-qa13:10
*** markus_z has joined #openstack-qa13:10
*** dpaterson has joined #openstack-qa13:11
*** mestery has joined #openstack-qa13:11
*** markus_z has quit IRC13:11
*** marun has quit IRC13:12
*** mestery has quit IRC13:13
*** mestery has joined #openstack-qa13:13
openstackrecheckopenstack/nova change: https://review.openstack.org/131461 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1373513 https://bugs.launchpad.net/bugs/135249813:14
*** marun_ is now known as marun13:15
chandankumardkranz, hello13:17
*** beagles_brb is now known as beagles13:17
chandankumardkranz, jordanP gmann please have a look https://review.openstack.org/#/c/131693/ , When you are free. Thanks :)13:18
*** mwagner_lap has joined #openstack-qa13:21
*** dustins has joined #openstack-qa13:22
*** mriedem has joined #openstack-qa13:23
*** bknudson has joined #openstack-qa13:24
*** ajo_ has quit IRC13:24
*** ajo_ has joined #openstack-qa13:25
*** ajo_ has quit IRC13:26
*** SridharG1 has joined #openstack-qa13:27
*** SridharG has quit IRC13:29
*** amotoki has quit IRC13:31
openstackgerritchandan kumar proposed a change to openstack/tempest: Fixes volume client in volume tests  https://review.openstack.org/13169313:32
yfrieddkranz: is there a qa mtg tomorrow?13:33
*** nikesh_vedams_ has quit IRC13:33
dkranzyfried: I am not sure, but I won't be at it if there is.13:34
yfriedmtreinish: ^^13:34
yfrieddkranz: also, I heard you used to have a qa get-together at the beginning of previous summits. is there one next week? I didn't see anything in the ML13:35
dkranzyfried: no, it was not at the beginning. It was more spur-of-the-moment. The presence of the mid-cycle like event at the end lessens the need for this.13:36
yfrieddkranz: too bad I was looking forward to this. and I won't be at the final 2days13:37
dkranzyfried: sorry13:37
yfrieddkranz: you should be. :)13:40
*** yamahata has quit IRC13:41
*** yamahata has joined #openstack-qa13:41
*** cdent has quit IRC13:43
*** Krast has quit IRC13:43
*** ramishra has joined #openstack-qa13:45
*** rbak has quit IRC13:46
openstackgerritDavid Kranz proposed a change to openstack/qa-specs: Add spec for clients returning one value  https://review.openstack.org/12600413:46
*** cdent has joined #openstack-qa13:47
*** rushiagr is now known as rushiagr_away13:47
*** toure has joined #openstack-qa13:50
yfrieddkranz: about your spec. I didn't understand your comment re my objection to inheriting dict13:50
dkranzyfried: I don't see a problem with inheriting. If we do what you are suggesting, we are relying on knowing what all the code inside unittest thinks this object is.13:53
dkranzyfried: The failures were in assertIn13:53
dkranzyfried: buried way down in some path13:53
*** malini has joined #openstack-qa13:53
dkranzyfried: I don't think this is a big deal and am not willing to invest more energy in this.13:53
yfrieddkranz: I just want to understand this13:54
dkranzyfried: I don't understand all the details13:54
yfrieddkranz: why is unittest a factor? we are replacing a normal dict with our own object13:54
dkranzyfried: yes, but we are passing that object to assertIn13:55
dkranzyfried: and that works when we "are" a dict, but not when we wrap it13:55
openstackgerritA change was merged to openstack-dev/grenade: Add comment explaining where javelin logs are found  https://review.openstack.org/11958713:55
yfrieddkranz: oh, I see - assertIn is designed to deal with dict but we'll be sending it the object13:55
dkranzyfried: yes13:55
dkranzyfried: actually I don't know whether it is designed to deal only with dict but the interface is unadvertised and therefore very fragile for us to depend on13:56
dkranzyfried: I agree with you in principal13:56
dkranzprinciple13:56
yfrieddkranz: ok. so please document that in the bp and the code to explain the choice of inheritance over composition13:56
dkranzyfried: I will document it in the bp13:57
yfrieddkranz: as I assume this question will be raised by others from time to time.13:57
dkranzyfried: well, AttributeDict also inherits from dict13:57
dkranzyfried: but ok13:58
yfrieddkranz: AttributeDict is what got me to object to your way in the first place13:59
yfrieddkranz: please add the reason the the code as well. Too much documentation is not a problem for us ATM13:59
dkranzyfried: I am just saying that I don't expect code to be documented with every alternative implementation choice that could have been made.13:59
yfrieddkranz: me too, but I forsee this would come into question from time to time and havine a one-line about the assertIn as example would help a lot for ppl looking at it who weren't part of the spec discussion14:01
yfrieddkranz: need to go assemble puzzles. will talk later14:01
dkranzyfried: I really don't see what is wrong with inheriting from dict and don't see evidence that your counter-view is widely held.14:03
*** ramishra has quit IRC14:04
*** ramishra has joined #openstack-qa14:05
*** zz_dimtruck is now known as dimtruck14:08
openstackgerritDavid Kranz proposed a change to openstack/tempest: Add code to enable single-value response for http clients  https://review.openstack.org/13064514:09
*** ramishra has quit IRC14:10
*** gordc has joined #openstack-qa14:10
*** andreaf has quit IRC14:11
*** andreaf has joined #openstack-qa14:12
*** markmcclain has quit IRC14:12
openstackgerritA change was merged to openstack-dev/devstack: Use updated trove mysql image location  https://review.openstack.org/13100214:16
*** neeti has quit IRC14:17
*** rbak has joined #openstack-qa14:17
mroddenianw: sdague: just got done with my morning meetings and was going to check those patches from yesterday before doing a release on bashate14:18
mtreinishyfried: yes, I was planning to have the regular meeting tomorrow, even if its short14:20
*** mkollaro has joined #openstack-qa14:21
*** yfried is now known as yfried|afk14:21
*** zz_jgrimm is now known as jgrimm14:22
dkranzyfried|afk: BTW, I'm not sure where the idea that inheriting from dict is bad comes from. It is even advertised as a feature here: https://docs.python.org/2/library/userdict.html14:29
mtreinishdkranz: there's nothing wrong with doing it14:30
*** SridharG1 has left #openstack-qa14:30
mroddenianw: sdague: https://pypi.python.org/pypi/bashate/0.3.014:30
dkranzmtreinish: :)14:30
sdaguemrodden: great14:30
*** hop has joined #openstack-qa14:30
sdaguemrodden: is there release instructions somewhere?14:31
mroddenyeah somewhere...14:31
mroddenlet me look14:31
mtreinishmrodden: https://wiki.openstack.org/wiki/Nova/Client_Release_Process14:32
mtreinishit's just create a tag and then git push gerrit tag14:32
mtreinishmrodden: or https://wiki.openstack.org/wiki/QA/releases14:33
*** yfried|afk is now known as yfried14:33
mroddeni usually follow this https://wiki.openstack.org/wiki/GerritJenkinsGit#Tagging_a_Release14:37
mroddenits a bit more generic to all projects14:37
mroddenexcept i tagged the wrong thing14:40
mroddenderp14:40
mrodden0.3.1 out now :)14:43
*** neeti has joined #openstack-qa14:45
*** yfried is now known as yfried|afk14:45
*** mpaolino has quit IRC14:45
*** Yogi2 has joined #openstack-qa14:53
*** neeti has quit IRC14:53
*** Yogi has quit IRC14:55
*** yamahata has quit IRC14:58
*** yamahata has joined #openstack-qa14:58
*** mpaolino has joined #openstack-qa14:59
*** jaypipes is now known as jaypipes-afk15:00
*** Yogi has joined #openstack-qa15:02
*** Yogi2 has quit IRC15:02
*** rushiagr_away is now known as rushiagr15:08
*** yfried has joined #openstack-qa15:09
*** neeti has joined #openstack-qa15:10
*** yfried_ has joined #openstack-qa15:12
*** yfried|afk has quit IRC15:12
*** pcrews has joined #openstack-qa15:12
*** yamamoto has joined #openstack-qa15:13
*** yfried has quit IRC15:15
*** yfried_ has quit IRC15:22
openstackgerritJohn Davidge proposed a change to openstack-dev/devstack: Add IPv6 support for tenant data network  https://review.openstack.org/8798715:22
*** hemna__ is now known as hemna15:30
*** atiwari has joined #openstack-qa15:36
*** mlavalle has joined #openstack-qa15:38
*** Sukhdev has joined #openstack-qa15:39
*** oomichi has joined #openstack-qa15:41
*** oomichi has quit IRC15:42
*** patrickeast has joined #openstack-qa15:47
*** mkollaro has quit IRC15:57
dkranzchmouel: when deploying swift, is it required to create the ResellerAdmin role?15:57
*** sergsh has left #openstack-qa15:57
dkranzchmouel: tempest adds this role in one of its tests and I was wondering if it is the responsibility of a swift deployer to create that role.15:58
dkranznotmyname: ^^^15:58
*** evgenyf has quit IRC16:02
*** neeti has quit IRC16:03
*** k4n0 has quit IRC16:03
*** chandankumar has quit IRC16:03
*** ajo_ has joined #openstack-qa16:05
*** Poornima has quit IRC16:06
*** mwagner_lap has quit IRC16:08
mtreinishdkranz: IIRC, that test is testing the use of that role which is why it adds the role from config to a user16:08
*** Yogi has quit IRC16:09
mtreinishdkranz: I think it get's set by default in the swift keystone auth to 'ResellerAdmin' so it should always be set (I'm not 100% though it's been a while since I looked at it)16:10
*** ajo_ has quit IRC16:16
*** Yogi has joined #openstack-qa16:18
*** markmcclain has joined #openstack-qa16:18
*** marun has quit IRC16:24
*** SridharGaddam has quit IRC16:27
dkranzmtreinish: I understand that and it implies that swift is using that role. The question was whether a swift deployment is buggy if it does not create that role in keystone.16:35
*** zhiyan has quit IRC16:36
dkranzmtreinish: This is part of the project to separate tempest from devstack or any particular installer completely.16:36
*** tosky has joined #openstack-qa16:37
mtreinishdkranz: no, that's what I was saying I think when you configure to swift to use keystone auth it creates the role16:38
dkranzmtreinish: so you are saying the answer is yes, when you install swift you are supposed to do that. So tempest configuration does not need to deal with that.16:39
dkranzmtreinish: That is what I thought but wanted confirmation16:40
mtreinishdkranz: well assuming you've configured swift to use keystone auth (which is a prereq for tempest)16:40
mtreinishif you give me a few mins I can take a look in the code16:40
dkranzmtreinish: I can look. Which code?16:41
mtreinishthe swift keystone auth bits16:42
dkranzmtreinish: I checked. There is a reseller_admin_role in conf that defaults to ResellerAdmin. So if you deploy swift/keystone and don't create that role, your install procedure is buggy. That is what I was looking for.16:45
dkranzmtreinish: I mean in swift conf file16:46
mtreinishyeah, I knew what you meant :)16:46
openstackgerritA change was merged to openstack/tempest: Fix the flavor ram string value  https://review.openstack.org/13163716:48
openstackrecheckopenstack/glance change: https://review.openstack.org/128928 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/137351316:49
chmoueldkranz: yes, we need the ResellerAdmin for some of the tempest tests,16:50
dkranzmtreinish: So I just read your email and it represents my thoughts exactly. Do you want to co-lead that cross-project session?16:51
chmoueldkranz: not for every use case but to test the access to different projects from the same admin user16:51
dkranzchmouel: yes, thanks16:51
dkranzmtreinish: BTW, it would be great if we could use changes to the clients in tempest-lib as the way to detect changes.16:52
dkranzmtreinish: But I think we will need to get rid of usage of **kwargs as inputs to the rest client functions in order to do that.16:53
mtreinishdkranz: I'm fine letting you lead it, the leader just sits in the center :)16:53
dkranzmtreinish: else you could change the api but not the code since it would be a pass-through16:53
dkranzmtreinish: ok :)16:53
dkranzmtreinish: but I am glad to see we are on the same page since that was not clear to me in the past16:54
mtreinishdkranz: heh, I'm sure when we get down to the details there will be differences in opinion :)16:56
*** hop has quit IRC16:59
*** suneelb has joined #openstack-qa16:59
openstackrecheckopenstack/ceilometer change: https://review.openstack.org/131525 failed because of: gate-tempest-dsvm-neutron-full: https://bugs.launchpad.net/bugs/137049617:11
openstackgerritMate Lakat proposed a change to openstack-dev/devstack: XenAPI: move Neutron plugin install to domU  https://review.openstack.org/12920817:12
openstackgerritMate Lakat proposed a change to openstack-dev/devstack: XenAPI: Remove non-used functions  https://review.openstack.org/12920917:12
openstackgerritMate Lakat proposed a change to openstack-dev/devstack: XenAPI: workaround for unsupp. associative arrays  https://review.openstack.org/12760317:12
*** mwagner_lap has joined #openstack-qa17:12
*** yamamoto has quit IRC17:15
*** harlowja_away is now known as harlowja17:16
*** jpich has quit IRC17:16
*** jlibosva has quit IRC17:20
*** Poornima has joined #openstack-qa17:21
*** harlowja_ has joined #openstack-qa17:21
*** harlowja has quit IRC17:21
*** achanda has joined #openstack-qa17:23
openstackgerritMatt Riedemann proposed a change to openstack/tempest: POC: test if unicode server name works on boot  https://review.openstack.org/13183317:23
*** e0ne has quit IRC17:28
*** Sukhdev has quit IRC17:28
*** markmcclain has quit IRC17:31
*** jaypipes-afk is now known as jaypipes17:34
*** achanda has quit IRC17:35
*** andreaf has quit IRC17:35
*** andreaf has joined #openstack-qa17:35
*** qba73 has quit IRC17:36
mriedemmtreinish: grenade testing with neutron is non-voting/experimental right now right?17:37
*** jamielennox is now known as jamielennox|away17:40
mtreinishmriedem: yes17:41
mtreinishit was non-voting check for a bit, but we moved it to the experimental queue17:41
mriedemmtreinish: ok, going up on the CI etherpad gap list then :)17:42
mtreinishmriedem: link?17:43
mriedemit's not ready yet :)17:43
mriedemi'll present it in all it's glory soon17:44
*** beekneemech has joined #openstack-qa17:44
mtreinishheh, ok17:44
mriedemi'm letting mastodon at high volumes move my fingeres17:44
*** bnemec has quit IRC17:44
*** achanda has joined #openstack-qa17:48
*** skata_ has joined #openstack-qa17:49
skata_Hello .. I get the follwoing error when gerrit tries to connect to review.openstack.org " Connection error : com.jcraft.jsch.JSchException: ProxyHTTP: java.io.IOException: proxy error: Forbidden" .. Can anyone give me pointers to the debug the issue? Any help is really appreciated! Thanks!17:50
*** achanda has quit IRC17:51
*** aix has quit IRC17:52
*** yamahata has quit IRC17:53
*** harlowja_ has quit IRC17:55
*** tkammer_ has quit IRC17:55
*** harlowja has joined #openstack-qa17:56
*** afazekas has quit IRC17:57
*** zhiyan has joined #openstack-qa18:00
openstackgerritA change was merged to openstack/tempest: Correct docstring indentation  https://review.openstack.org/13136618:01
*** pcrews has quit IRC18:07
*** safchain has quit IRC18:07
* cdent tries devstack on f2118:09
*** jordanP has quit IRC18:09
*** tosky has quit IRC18:16
*** andreaf has quit IRC18:17
*** tosky has joined #openstack-qa18:18
*** markmcclain has joined #openstack-qa18:25
*** Poornima has quit IRC18:28
*** vrovachev has quit IRC18:31
*** gszasz has quit IRC18:33
*** mpaolino has quit IRC18:43
lifelessrandom question18:45
lifelessis anyone interested in validating the unittest2 changes before I do a 0.7.0 release containing them ?18:45
*** skata_ has quit IRC18:48
mordredhrm18:48
* mordred trying to think of a non-insane way to test that18:48
lifelessits only two years of backports of unittest changes in cpython that we're not using anywhere else.18:48
lifelessWHAT COULD POSSIBLY GO WRONG18:48
mordredlifeless: I can't think of a thing18:51
*** achanda has joined #openstack-qa18:52
*** johnthetubaguy is now known as zz_johnthetubagu18:54
*** eglynn has quit IRC18:57
*** achanda has quit IRC18:57
*** markmcclain1 has joined #openstack-qa19:01
openstackgerritClark Boylan proposed a change to openstack-infra/devstack-gate: Enable nova multi host on aiopcpu tests  https://review.openstack.org/13160419:01
*** markmcclain2 has joined #openstack-qa19:02
*** sabeen1 has joined #openstack-qa19:02
*** sabeen2 has quit IRC19:03
*** markmcclain has quit IRC19:03
*** sabeen1 has quit IRC19:06
*** markmcclain1 has quit IRC19:06
*** sabeen1 has joined #openstack-qa19:06
*** pcrews has joined #openstack-qa19:07
openstackgerritA change was merged to openstack-dev/devstack: allow for soft updating of global-requirements  https://review.openstack.org/13119819:08
*** rushiagr is now known as rushiagr_away19:10
*** Sukhdev has joined #openstack-qa19:13
*** tosky has quit IRC19:14
openstackgerritClark Boylan proposed a change to openstack-infra/devstack-gate: Enable nova multi host on aiopcpu tests  https://review.openstack.org/13160419:19
*** __afazekas is now known as afazekas19:30
sdaguedtroyer: if you aren't on a plane already - what's valid values for endpoint_type except publicURL?19:31
sdaguehttps://review.openstack.org/#/c/129357/4/novaclient/shell.py19:31
dtroyersdague:  (between flights)  adminURL and internalURL IIRC19:32
sdagueok19:32
sdagueseems odd that we don't have the service catalog set the preference, as it should be aware of what works for the environment, no?19:33
mtreinishsdague: I think public, admin, and internal are also valid values too19:33
sdaguemtreinish: ok, is there an enum somewhere? I did moderate grepping and just found strings being passed around19:34
*** malini has quit IRC19:34
mtreinishsdague: heh, I just remembered when it was added to tempest and looked at the list of what's allowed there19:34
mtreinishI would imagine it's somewhere in keystone? lbragstad ^^^ any ideas?19:35
*** cdent has quit IRC19:35
*** eglynn has joined #openstack-qa19:35
openstackgerritClark Boylan proposed a change to openstack-infra/devstack-gate: Enable nova multi host on aiopcpu tests  https://review.openstack.org/13160419:36
*** sabeen1 has quit IRC19:38
dtroyersdague: before the auth plugins that was all a mess, I haven't looked at cleaning up the old clients as they get updated, not sure if that's possible without breaking compat19:38
*** mkollaro has joined #openstack-qa19:38
*** sabeen1 has joined #openstack-qa19:40
*** dimtruck is now known as zz_dimtruck19:46
*** zz_dimtruck is now known as dimtruck19:46
*** beekneemech is now known as bnemec19:48
*** markmcclain2 has quit IRC19:50
*** markmcclain has joined #openstack-qa19:51
openstackgerritClark Boylan proposed a change to openstack-infra/devstack-gate: Enable nova multi host on aiopcpu tests  https://review.openstack.org/13160419:51
openstackgerritSean Dague proposed a change to openstack-dev/devstack: Add pcre devel package for Horizon  https://review.openstack.org/12860319:51
*** pcrews has quit IRC19:53
*** achanda has joined #openstack-qa19:59
*** yfried_ has joined #openstack-qa20:06
*** yfried_ is now known as yfried|afk20:06
*** aysyd has quit IRC20:11
*** markmcclain has quit IRC20:13
*** yfried has joined #openstack-qa20:18
lbragstadmtreinish: reading back..20:18
*** yfried|afk has quit IRC20:18
lbragstadmtreinish: looking for what is allowed as an acceptable value for endpoint['interface']?20:19
lbragstadmtreinish: https://github.com/openstack/keystone/blob/f45b3e5e0000f33c3da0349a0f475d5de71ee9de/keystone/catalog/schema.py#L7120:21
lbragstadsdague: ^20:21
lbragstadso, 'admin', 'internal', and 'public' are acceptable values20:21
sdaguelbragstad: ok, so publicURL ?20:21
sdagueor are we munging that somewhere along the way20:21
lbragstadsdague: amending the above statement for V3 only..20:22
lbragstadsdague: checking V220:22
*** yfried_ has joined #openstack-qa20:23
*** yfried_ is now known as yfried|afk20:23
*** pcrews has joined #openstack-qa20:23
*** yfried has quit IRC20:24
lbragstadsdague: according to the v2 api code, 'publicurl' is mandatory for create endpoint requests. https://github.com/openstack/keystone/blob/f45b3e5e0000f33c3da0349a0f475d5de71ee9de/keystone/catalog/controllers.py#L9820:24
*** yfried|afk has quit IRC20:25
*** adam_g_gone is now known as adam_g20:30
*** atiwari has quit IRC20:35
*** afazekas_ has joined #openstack-qa20:36
openstackrecheckopenstack/nova change: https://review.openstack.org/114539 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/124445720:45
*** mkollaro has quit IRC20:45
*** achanda has quit IRC20:50
*** patrickeast has quit IRC20:59
openstackgerritJoe Gordon proposed a change to openstack-dev/devstack: Make allow_{resize|migrate}_to_same_host configurable  https://review.openstack.org/13155720:59
*** patrickeast has joined #openstack-qa20:59
sdaguejogo: so why != "False" vs. == "True"21:03
sdaguedouble negatives are a little weird to read21:03
*** patrickeast_ has joined #openstack-qa21:03
sdagueif you respin fast I'll give you a +2 :)21:03
*** patrickeast has quit IRC21:03
*** patrickeast_ is now known as patrickeast21:03
jogosdague: I cut and paste from somewhere else21:05
*** Sukhdev has quit IRC21:05
openstackgerritJoe Gordon proposed a change to openstack-dev/devstack: Make allow_{resize|migrate}_to_same_host configurable  https://review.openstack.org/13155721:05
jogosdague: ^ fixed21:05
sdague+221:06
*** toure has quit IRC21:06
*** dpaterson has quit IRC21:06
mtreinishjogo: I'll +A it when jenkins comes back21:06
mtreinishalthough I prefer double negatives...21:06
*** Yogi has quit IRC21:07
jogomtreinish: hehe21:09
mtreinishsdague: what to +A the tempest-lib migration patch: https://review.openstack.org/11764921:11
*** dimtruck is now known as zz_dimtruck21:15
*** weshay has quit IRC21:15
sdaguemtreinish: sure, though it did fail again.21:15
openstackgerritJim Rollenhagen proposed a change to openstack-dev/devstack: Add iptables rule for Ironic VMs -> Swift  https://review.openstack.org/13187621:16
*** JayJ has joined #openstack-qa21:17
*** dkranz has quit IRC21:17
mtreinishsdague: ooh, a 500 on a server get that's ugly: http://logs.openstack.org/49/117649/11/check/check-tempest-dsvm-neutron-full/f59d991/console.html#_2014-10-27_16_16_23_31321:18
*** zz_dimtruck is now known as dimtruck21:18
jogojust got a ERROR (InstanceInErrorState): No valid host was found.21:19
jogofrom HP21:19
*** marun has joined #openstack-qa21:19
mtreinishjogo: heh, I've seen that from hp cloud before too21:19
mroddenruh roh21:19
openstackgerritJim Rollenhagen proposed a change to openstack-dev/devstack: Add iptables rule for Ironic VMs -> Swift  https://review.openstack.org/13187621:19
jogomoar cloud please21:19
sdagueok, time to call it an day. Will chat with folks in the morning21:20
*** achanda has joined #openstack-qa21:24
*** ajo_ has joined #openstack-qa21:25
*** patrickeast has quit IRC21:25
*** dustins has quit IRC21:30
*** patrickeast has joined #openstack-qa21:32
*** patrickeast has quit IRC21:34
*** patrickeast has joined #openstack-qa21:35
*** gordc has quit IRC21:41
mtreinishjogo, mriedem: ugh, a timeout to neutron: http://logs.openstack.org/49/117649/11/check/check-tempest-dsvm-neutron-full/f59d991/logs/screen-n-api.txt.gz#_2014-10-27_16_05_16_23921:42
mriedemmtreinish: i feel like that's not news...21:43
mtreinishprobably not, I'm sure it's been seen before21:43
mriedemi've seen that kind of thing a lot in our internal CI runs21:44
mriedemwith smaller flavor'ed vm's21:44
mriedem2 cpu21:44
mriedembut not running concurrently with testr21:45
mriedemwe have what now, ncpu / 2 workers in devstack for everything except keystone workers?21:45
mtreinishmriedem: yeah I think that's right. jogo ^^^?21:47
mtreinishmriedem: but if you're running serially that would be weird because it shouldn't have an issue handling 1 request at a time21:47
jogomtreinish: sounds right21:48
mtreinishmriedem: hmm, looking at the neutron log it doesn't crash or anything but there is a 30sec gap where the request fails: http://logs.openstack.org/49/117649/11/check/check-tempest-dsvm-neutron-full/f59d991/logs/screen-q-svc.txt.gz#_2014-10-27_16_05_16_23821:48
jogomtreinish: check dstat21:48
mtreinishjogo: low load avgs ~90% idle21:49
jogothat will even show things like failing hard disks (had one recently myself)21:49
jogomtreinish: even around that 30 second gap?21:49
jogolook at disk IO21:49
mtreinishoops wrong timestamp21:49
jogoand amount of CPU waiting for disk21:49
*** openstackgerrit has quit IRC21:50
mriedem41 hits in 7 days http://goo.gl/y6AH0r21:51
mriedemnot sure if we have a bug written21:51
mtreinishjogo: http://paste.openstack.org/show/126725/21:51
mriedemwhy is java running again?21:52
mtreinishmriedem: I think that's the jenkins slave stuff21:53
mriedemah21:53
mtreinishmriedem: yeah it is: http://logs.openstack.org/49/117649/11/check/check-tempest-dsvm-neutron-full/f59d991/logs/ps.txt.gz21:53
jogomriedem: yeah jenkins21:53
mtreinishalthough I have no idea why that would be the top process21:53
jogo60% WAIT is a lot I think21:53
mtreinishclarkb: ^^^21:53
jogomtreinish: looks like the disk stops responding for a bit21:54
jogomtreinish: we have seen that before21:54
jogomtreinish: I fixed that at home by replacing my HD over the weekend21:54
mtreinishjogo: so you just think it's a bad node or something?21:54
jogomtreinish: that is my working theory21:55
jogoI wonder what happens if we try using tmfs for some of the more heavily used directories21:55
jogotmpfs*21:55
mtreinishjogo: well, don't we only have ~2gb avail during a normal run is that enough space?21:57
jogomtreinish: prob not21:57
jogoI think we have a bit more then that but not a lot of space21:57
mtreinishoh, but on the hp nodes I think clarkb said we're using a 32gb flavor and just only use 821:58
jogowe use 6+GB of what appears t obe 821:58
mtreinishso maybe we could play with it there21:58
jogomtreinish: I don't think it would help21:58
jogoits more of a fun bandaid21:58
jogobut it sounds like we are disk limited in a lot of runs21:58
jogowe should figure out of this is a RAX or HP thing or both21:59
clarkbits 8gb of 30gb nodes22:00
clarkbto ensure we dont actually need 30gb22:00
mtreinishclarkb: is there any way we could turn the unused into a tmpfs?22:01
clarkbthen we would need 30gb22:02
clarkbI would bring it up in infra22:02
mtreinishok22:03
*** achanda_ has joined #openstack-qa22:04
jogoclarkb: the idea would be, do a lot of disk IO, and this may be a way to speed up tests22:08
jogotmpfs allows swapping etc22:08
jogoclarkb: it may not be worth it though22:08
*** achanda has quit IRC22:08
clarkbya but then we require 30gb nodes to reproduce the gate22:08
clarkband our nodes are not that big in rax22:09
jogomtreinish: random question22:16
jogomtreinish: is keystone v3 the default for nova tempest tests?22:17
jogomtreinish: if not how do we change that?22:17
jogomtreinish: as it *should* just work at this point22:17
mtreinishjogo: it's not, tempest by default will get a token using keystone v2 right now22:18
*** suneelb has quit IRC22:18
mtreinishthere is an experimental job which runs using keystone v322:18
jogomtreinish: oh cool, whats the timeline to make that the default?22:18
jogoand does it work?22:19
*** suneelb has joined #openstack-qa22:19
mtreinishjogo: don't know if it works, it was blocked for a while because of the official client story22:19
jogoright22:19
mtreinishbut since we moved to using the tempest clients for scenario that's not a problem anymore22:19
jogomtreinish: any good patch to test it on?22:19
mtreinishjogo: it looks like it works: http://logs.openstack.org/55/130655/3/experimental/check-tempest-dsvm-keystonev3-full/39d292d/22:20
jogooh I'll just use https://review.openstack.org/#/c/117649/ :)22:20
jogois the format to do experimental still 'check experimental'?22:21
mtreinishjogo: it is, but those results I posted are from monday22:21
jogomtreinish: cool, assuming we didn't break it since Monday22:22
jogoI think we should try to move over to v3 by default in the not to distant future22:22
mtreinishI'd hope so :)22:22
jogomtreinish: maybe let morganfainberg push a patch up22:23
jogomorganfainberg: ^22:23
mtreinishalso the cli tests will still be using v2 because there's nothing else for them22:23
jogomtreinish: right, well that will be fixed in nova soon22:23
mtreinishwell probably not on the tempest test side22:24
jogomtreinish: yeah ... true22:24
mtreinishjogo: but it wouldn't be too hard to do, after 117649 merges I'll push a patch up to make the auth layer for the cli tests more consistent22:25
mtreinishbut it'll still be v2 on the cli I'm not sure how that works22:25
*** bknudson has quit IRC22:26
*** dims_ has joined #openstack-qa22:28
*** dims_ has quit IRC22:29
*** dims_ has joined #openstack-qa22:30
*** dims__ has quit IRC22:31
morganfainbergjogo: sure will be happy to push a patch.22:32
morganfainbergBut it won't be till tonight or tomorrow.22:33
*** achanda_ has quit IRC22:34
jogomtreinish: ^ read backlog with mtreinish22:37
mtreinishjogo: read the backlog with myself?22:38
jogomtreinish: derp22:39
jogomorganfainberg: ^22:39
*** rbak has quit IRC22:55
*** mriedem has quit IRC22:55
*** hemna is now known as hemna__22:57
*** achanda has joined #openstack-qa22:58
openstackrecheckopenstack/nova change: https://review.openstack.org/129199 failed because of: gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/1373513 https://bugs.launchpad.net/bugs/1376941 https://bugs.launchpad.net/bugs/135249822:58
*** Yogi has joined #openstack-qa22:59
*** Yogi2 has joined #openstack-qa23:02
jogomtreinish: ^ 323:04
jogogood to have openstackrecheck boack23:04
*** Yogi has quit IRC23:04
*** aysyd has joined #openstack-qa23:05
*** dims_ has quit IRC23:05
*** dims__ has joined #openstack-qa23:06
*** gfidente has quit IRC23:06
*** ajo_ has quit IRC23:22
*** ajo_ has joined #openstack-qa23:23
*** JayJ has quit IRC23:31
*** dimtruck is now known as zz_dimtruck23:41
*** annegentle has quit IRC23:42
*** annegentle has joined #openstack-qa23:42
*** ajo_ has quit IRC23:43
*** ajo_ has joined #openstack-qa23:44
*** annegentle has quit IRC23:46
*** annegentle has joined #openstack-qa23:47
*** annegentle has quit IRC23:48
*** annegentle has joined #openstack-qa23:48
*** annegentle has quit IRC23:49
*** annegentle has joined #openstack-qa23:49
*** jecarey has quit IRC23:56
*** bknudson has joined #openstack-qa23:56
*** asselin has quit IRC23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!