*** inara` has quit IRC | 00:00 | |
mriedem | i wonder if we could bulk query the floating IPs by the list of fixed IPs and the port_id | 00:01 |
---|---|---|
openstackgerrit | melanie witt proposed openstack/nova master: Cache database and message queue connection objects https://review.openstack.org/465042 | 00:01 |
mriedem | "When you specify multiple filters, the Networking API v2.0 returns only objects that meet all filtering criteria. The operation applies an AND condition among the filters." | 00:02 |
mriedem | so that wouldn't really work, | 00:02 |
mriedem | because fixed_ip1 AND fixed_ip2 wouldn't resolve to a floating IP | 00:02 |
*** inara has joined #openstack-nova | 00:02 | |
cfriesen | mriedem: we get an event for each vif, and we have to get info for all vifs on each event, so when we have N vifs, the live-migration overhead goes up as N^2 | 00:02 |
mriedem | we could trim the fat on the fields we actually need to get back | 00:02 |
cfriesen | mriedem: it'd be better to have the vif specified in the network-changed event | 00:03 |
*** ijw has quit IRC | 00:03 | |
cfriesen | mriedem: and avoid having to scan all the vifs | 00:03 |
mriedem | cfriesen: yeah i'm not sure how feasible that is from the neutron side | 00:03 |
mriedem | i guess it would depend on how they calculate the network-changed event | 00:03 |
cfriesen | mriedem: gotta run, family is calling for supper | 00:03 |
edleafe | I've written a post to the dev list about continuing the discussion on claims in the scheduler: http://lists.openstack.org/pipermail/openstack-dev/2017-May/116949.html | 00:03 |
edleafe | ^^ cdent jaypipes dansmith bauzas johnthetubaguy | 00:04 |
edleafe | It has a link to my blog post and the patch series | 00:04 |
mriedem | edleafe: ack | 00:04 |
edleafe | And with that, I'm off to PyCon | 00:04 |
mriedem | have fun | 00:04 |
edleafe | See you all next week! | 00:04 |
edleafe | mriedem: oh, it's all work work work | 00:04 |
*** suresh12 has quit IRC | 00:10 | |
*** suresh12 has joined #openstack-nova | 00:13 | |
*** edleafe has quit IRC | 00:13 | |
*** edleafe has joined #openstack-nova | 00:14 | |
mriedem | cfriesen: here is something to look into later - the network-changed event in neutron does have a port in scope, it just doesn't put it in the external event request https://github.com/openstack/neutron/blob/master/neutron/notifiers/nova.py#L161 | 00:15 |
mriedem | but we should be able to scope that event to the single vif | 00:15 |
mriedem | cfriesen: it would be helpful if we had a bug to track some of this | 00:16 |
*** ericyoung has quit IRC | 00:19 | |
*** ericyoung has joined #openstack-nova | 00:21 | |
*** gianpietro has quit IRC | 00:25 | |
*** jamielennox is now known as jamielennox|away | 00:27 | |
*** ijw has joined #openstack-nova | 00:31 | |
*** ijw has quit IRC | 00:31 | |
*** ijw has joined #openstack-nova | 00:32 | |
*** nic has quit IRC | 00:33 | |
*** suresh12 has quit IRC | 00:34 | |
*** suresh12 has joined #openstack-nova | 00:35 | |
mriedem | cfriesen: https://bugs.launchpad.net/neutron/+bug/1691602 for tracking | 00:41 |
openstack | Launchpad bug 1691602 in OpenStack Compute (nova) "live migration generates several network-changed events which lock up refreshing the nw info cache" [Undecided,New] | 00:41 |
*** jamielennox|away is now known as jamielennox | 00:41 | |
*** Sukhdev has joined #openstack-nova | 00:43 | |
*** Guest2661 has quit IRC | 00:45 | |
*** ijw has quit IRC | 00:48 | |
*** ijw has joined #openstack-nova | 00:49 | |
*** ijw has quit IRC | 00:49 | |
*** ijw has joined #openstack-nova | 00:50 | |
*** zz_dimtruck is now known as dimtruck | 00:50 | |
*** ijw has quit IRC | 00:51 | |
*** ijw has joined #openstack-nova | 00:52 | |
*** ijw has quit IRC | 00:53 | |
*** ijw has joined #openstack-nova | 00:53 | |
*** gianpietro has joined #openstack-nova | 00:56 | |
*** thorst_afk has joined #openstack-nova | 00:58 | |
*** ijw has quit IRC | 00:58 | |
openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Add policy description for os-host https://review.openstack.org/465458 | 00:59 |
*** rfolco has joined #openstack-nova | 01:00 | |
*** rfolco has quit IRC | 01:01 | |
*** Apoorva_ has joined #openstack-nova | 01:01 | |
*** thorst_afk has quit IRC | 01:02 | |
*** Apoorva has quit IRC | 01:05 | |
*** Sukhdev has quit IRC | 01:06 | |
*** Apoorva_ has quit IRC | 01:06 | |
*** kaisers_ has joined #openstack-nova | 01:10 | |
*** Kevin_Zheng has joined #openstack-nova | 01:11 | |
*** jaypipes has joined #openstack-nova | 01:12 | |
*** kaisers has quit IRC | 01:14 | |
*** tuanluong has joined #openstack-nova | 01:15 | |
*** cNilesh has joined #openstack-nova | 01:15 | |
*** felipemonteiro has joined #openstack-nova | 01:15 | |
*** liverpooler has joined #openstack-nova | 01:15 | |
*** cNilesh has quit IRC | 01:16 | |
*** cnilesh has joined #openstack-nova | 01:16 | |
*** cnilesh has quit IRC | 01:17 | |
*** cNilesh has joined #openstack-nova | 01:17 | |
openstackgerrit | Alex Xu proposed openstack/nova master: Limit the min length of string for integer JSON-Schema https://review.openstack.org/415103 | 01:18 |
*** Daisy has joined #openstack-nova | 01:18 | |
*** jaypipes has quit IRC | 01:19 | |
*** karthiks has joined #openstack-nova | 01:20 | |
openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Catch exception.OverQuota when create image for volume backed instance https://review.openstack.org/464548 | 01:21 |
openstackgerrit | Alex Xu proposed openstack/nova master: Limit the min length of string for integer JSON-Schema https://review.openstack.org/415103 | 01:24 |
*** gjayavelu has quit IRC | 01:25 | |
*** jamesden_ has joined #openstack-nova | 01:26 | |
*** fragatina has quit IRC | 01:27 | |
*** ijw has joined #openstack-nova | 01:30 | |
*** Daisy has quit IRC | 01:32 | |
*** rfolco has joined #openstack-nova | 01:33 | |
*** ijw has quit IRC | 01:36 | |
dikonoor | pumaranikar: Hi | 01:36 |
dikonoor | pumaranikar: I am trying to understand if this new support added is still experimental > https://github.com/openstack/nova/blob/master/nova/conf/service_token.py#L43 | 01:39 |
dikonoor | Can it be turned on ? | 01:39 |
mriedem | dikonoor: yes it can be turned on for testing, we test with it on in the nova-next CI job | 01:40 |
mriedem | dikonoor: would actually appreciate someone to run with it in rally or longer-running operation scenarios like with live migration | 01:41 |
mriedem | and report back if it's helping | 01:41 |
*** thorst_afk has joined #openstack-nova | 01:41 | |
*** p01nt3r75 has joined #openstack-nova | 01:43 | |
*** jwcroppe has joined #openstack-nova | 01:43 | |
*** thorst_afk has quit IRC | 01:43 | |
*** jwcroppe has quit IRC | 01:44 | |
*** jwcroppe has joined #openstack-nova | 01:44 | |
*** psachin has joined #openstack-nova | 01:51 | |
*** baoli has quit IRC | 01:56 | |
*** baoli has joined #openstack-nova | 01:57 | |
*** suresh12 has quit IRC | 01:58 | |
*** coreywright has quit IRC | 02:05 | |
*** gianpietro has quit IRC | 02:08 | |
*** markvoelker has quit IRC | 02:08 | |
*** markvoelker has joined #openstack-nova | 02:08 | |
*** crushil has joined #openstack-nova | 02:10 | |
*** Daisy has joined #openstack-nova | 02:12 | |
*** gianpietro has joined #openstack-nova | 02:13 | |
*** yamahata has joined #openstack-nova | 02:15 | |
openstackgerrit | Jay Pipes proposed openstack/nova master: Ensure image metadata when unshelving BFV instance https://review.openstack.org/347063 | 02:16 |
*** hongbin has joined #openstack-nova | 02:18 | |
*** jaypipes has joined #openstack-nova | 02:22 | |
*** rfolco has quit IRC | 02:22 | |
jaypipes | alex_xu: looks sensible to me: https://review.openstack.org/#/c/353862/23 | 02:22 |
jaypipes | alex_xu: XenAPI resource tracking fix. | 02:23 |
*** coreywright has joined #openstack-nova | 02:23 | |
*** suresh12 has joined #openstack-nova | 02:23 | |
*** dimtruck has quit IRC | 02:28 | |
*** suresh12 has quit IRC | 02:28 | |
*** vladikr has quit IRC | 02:29 | |
*** dimtruck has joined #openstack-nova | 02:31 | |
openstackgerrit | Alex Xu proposed openstack/nova master: Correct the error message for query parameter validation https://review.openstack.org/465785 | 02:34 |
openstackgerrit | Tao Li proposed openstack/nova master: Fix the display of updated_at time when using memcache driver. https://review.openstack.org/414463 | 02:35 |
mriedem | oh bug smashing glory | 02:38 |
*** jamielennox is now known as jamielennox|away | 02:40 | |
*** jaypipes has quit IRC | 02:41 | |
mriedem | kevinbenton: looking at this https://github.com/openstack/nova/blob/master/nova/network/neutronv2/api.py#L331 | 02:43 |
mriedem | it looks like you can list resources by a list of ids | 02:43 |
*** crushil has quit IRC | 02:43 | |
kevinbenton | mriedem: yeah, i was actually just looking through the code that builds the filters | 02:44 |
kevinbenton | because i do remember that being a list somewhere | 02:44 |
*** thorst_afk has joined #openstack-nova | 02:44 | |
kevinbenton | mriedem: i don't remember if it's just for IDs or for everything | 02:44 |
mriedem | ok, because i was wondering if this could be optimized https://github.com/openstack/nova/blob/master/nova/network/neutronv2/api.py#L2017 | 02:44 |
mriedem | that's called from a for loop https://github.com/openstack/nova/blob/master/nova/network/neutronv2/api.py#L2097 | 02:45 |
*** gjayavelu has joined #openstack-nova | 02:45 | |
mriedem | if i could just do something like GET /floatingips?fixed_ip_address=['fixedip1', 'fixedip2', ...]&port_id=port_id | 02:45 |
mriedem | that would be the bulk query i'd need | 02:45 |
kevinbenton | mriedem: https://github.com/openstack/neutron/blob/master/neutron/db/_model_query.py#L191-L194 | 02:46 |
mriedem | but, | 02:46 |
kevinbenton | mriedem: yeah, it's an OR list | 02:46 |
*** baoli has quit IRC | 02:46 | |
*** gianpietro has quit IRC | 02:46 | |
*** gouthamr has quit IRC | 02:47 | |
mriedem | hmm, but the API ref says "OpenStack Networking does not offer an OR mechanism for filters." | 02:47 |
kevinbenton | mriedem: it wouldn't be the first time the docs were wrong :) | 02:47 |
mriedem | i figured the client would unpack the list and turn that into fixed_ip_address=1&fixed_ip_address=2&port_id=port_id | 02:47 |
mriedem | and AND them all | 02:47 |
*** gouthamr has joined #openstack-nova | 02:47 | |
*** salv-orlando has joined #openstack-nova | 02:48 | |
*** Shunli has joined #openstack-nova | 02:49 | |
*** Daisy has quit IRC | 02:49 | |
*** Daisy has joined #openstack-nova | 02:50 | |
*** jamesden_ has quit IRC | 02:50 | |
mriedem | ok i can post a wip for that too and see what happens, | 02:50 |
mriedem | i don't know if there are any tests in tempest which would tickle that | 02:50 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Handle network-changed event for a specific port https://review.openstack.org/465787 | 02:52 |
mriedem | kevinbenton: ^ is the kind of awful i was talking about earlier | 02:52 |
kevinbenton | mriedem: yeah, repeats turn into the list which becomes the OR in that first query link i sent | 02:53 |
kevinbenton | mriedem: here is the request parse https://github.com/openstack/neutron/blob/master/neutron/api/api_common.py#L46-L47 | 02:53 |
mriedem | cool | 02:53 |
*** gianpietro has joined #openstack-nova | 02:53 | |
kevinbenton | mriedem: yeah, not exactly a trivial change :) | 02:54 |
mriedem | it would really be split into 2 changes, the first being a refactor | 02:54 |
mriedem | but yeah | 02:54 |
mriedem | i'm not going for pretty at 10pm | 02:54 |
kevinbenton | mriedem: move to hawaii | 02:55 |
kevinbenton | more time in the day that way | 02:55 |
kevinbenton | :) | 02:55 |
mriedem | heh | 02:55 |
mriedem | you are a proven thought leader | 02:55 |
mriedem | apparently we handle GET /floatingips results as a list when we filter on fixed IP and port, | 03:00 |
mriedem | but wouldn't that really be 1:1? | 03:00 |
mriedem | can you have more than one floating IP associated to the same fixed ip and port? | 03:00 |
mriedem | i'd think not | 03:00 |
kevinbenton | no | 03:01 |
mriedem | ok good, this is just confusing nova code then | 03:01 |
kevinbenton | well let me confirm | 03:01 |
*** baoli has joined #openstack-nova | 03:02 | |
*** thorst_afk has quit IRC | 03:04 | |
kevinbenton | mriedem: there actually may be a way to get there | 03:04 |
mriedem | ew | 03:05 |
kevinbenton | mriedem: two different routers | 03:05 |
mriedem | ok | 03:05 |
kevinbenton | mriedem: attached to two different ext nets | 03:05 |
kevinbenton | mriedem: both attached to same internal | 03:05 |
kevinbenton | mriedem: and both with a floating IP associated to the instance | 03:05 |
kevinbenton | mriedem: and it would actually work okay if you setup some routing rules on the instance | 03:06 |
kevinbenton | mriedem: so i think you'll have to preserve that use case :) | 03:06 |
mriedem | damn | 03:06 |
mriedem | ok | 03:06 |
kevinbenton | mriedem: the constraint for floating IP is external_network + IP + port_id https://github.com/openstack/neutron/blob/master/neutron/db/models/l3.py#L101 | 03:07 |
*** gianpietro has quit IRC | 03:07 | |
*** nicolasbock has quit IRC | 03:09 | |
mriedem | i'll bake that into a comment in here | 03:09 |
*** felipemonteiro has quit IRC | 03:12 | |
*** trinaths has joined #openstack-nova | 03:12 | |
*** links has joined #openstack-nova | 03:14 | |
mriedem | if we only have one fixed ip per port then this doesn't buy us much | 03:16 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Bulk query floating IPs in _nw_info_get_ips https://review.openstack.org/465792 | 03:16 |
mriedem | but here is the thing ^ | 03:16 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Bulk query floating IPs in _nw_info_get_ips https://review.openstack.org/465792 | 03:17 |
*** crushil has joined #openstack-nova | 03:18 | |
*** baoli has quit IRC | 03:19 | |
openstackgerrit | Jay Pipes proposed openstack/nova master: Ensure image metadata when unshelving BFV instance https://review.openstack.org/347063 | 03:29 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Don't warn on expected network-vif-unplugged events https://review.openstack.org/465794 | 03:32 |
*** jamielennox|away is now known as jamielennox | 03:33 | |
*** salv-orlando has quit IRC | 03:34 | |
*** Daisy has quit IRC | 03:34 | |
*** Daisy has joined #openstack-nova | 03:34 | |
*** felipemonteiro has joined #openstack-nova | 03:35 | |
*** mriedem has quit IRC | 03:39 | |
*** Daisy has quit IRC | 03:39 | |
*** coreywright has quit IRC | 03:46 | |
*** gianpietro has joined #openstack-nova | 03:48 | |
*** yamamoto_ has joined #openstack-nova | 03:50 | |
*** vks1 has joined #openstack-nova | 03:50 | |
*** hongbin has quit IRC | 03:52 | |
*** gianpietro has quit IRC | 03:53 | |
openstackgerrit | Tao Li proposed openstack/nova master: Optimize the logged message https://review.openstack.org/465797 | 03:54 |
*** markvoelker has quit IRC | 03:56 | |
*** markvoelker has joined #openstack-nova | 03:57 | |
openstackgerrit | Luo Gangyi proposed openstack/nova master: convert unicode to string before we connect to rados https://review.openstack.org/465404 | 03:57 |
*** coreywright has joined #openstack-nova | 03:59 | |
cfriesen | mriedem: you've been a busy beaver on get_instance_nw_info()...taking a look at your patch. | 04:00 |
*** litao__ has joined #openstack-nova | 04:01 | |
*** markvoelker has quit IRC | 04:01 | |
*** ijw has joined #openstack-nova | 04:01 | |
*** gianpietro has joined #openstack-nova | 04:01 | |
*** suresh12 has joined #openstack-nova | 04:05 | |
*** ijw has quit IRC | 04:06 | |
*** gjayavelu has quit IRC | 04:10 | |
*** takashin has joined #openstack-nova | 04:10 | |
*** Sukhdev has joined #openstack-nova | 04:13 | |
*** suresh12 has quit IRC | 04:13 | |
*** armax has quit IRC | 04:14 | |
*** zerda3 has quit IRC | 04:14 | |
*** gianpietro has quit IRC | 04:18 | |
*** takashin has quit IRC | 04:19 | |
*** links has quit IRC | 04:20 | |
*** fragatina has joined #openstack-nova | 04:30 | |
*** fragatina has quit IRC | 04:30 | |
*** fragatina has joined #openstack-nova | 04:31 | |
*** takashin has joined #openstack-nova | 04:32 | |
openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove unused exceptions https://review.openstack.org/464525 | 04:32 |
openstackgerrit | Takashi NATSUME proposed openstack/nova master: Refactor a test method including 7 test cases https://review.openstack.org/395436 | 04:33 |
*** diablo_rojo has quit IRC | 04:34 | |
openstackgerrit | Takashi NATSUME proposed openstack/nova master: Avoid forcing translation on logging calls https://review.openstack.org/413876 | 04:35 |
*** prateek has joined #openstack-nova | 04:36 | |
*** Daisy has joined #openstack-nova | 04:36 | |
*** prateek has quit IRC | 04:38 | |
*** dave-mccowan has quit IRC | 04:43 | |
*** ayogi has joined #openstack-nova | 04:45 | |
*** nurdie has joined #openstack-nova | 04:47 | |
*** adisky_ has joined #openstack-nova | 04:50 | |
*** bkopilov has joined #openstack-nova | 04:51 | |
*** sree has joined #openstack-nova | 04:51 | |
*** crushil has quit IRC | 04:52 | |
*** Daisy has quit IRC | 04:53 | |
*** Daisy has joined #openstack-nova | 04:53 | |
*** crushil has joined #openstack-nova | 04:54 | |
*** yamahata has quit IRC | 04:54 | |
*** crushil has quit IRC | 04:56 | |
*** trinaths has left #openstack-nova | 04:57 | |
*** yamahata has joined #openstack-nova | 04:58 | |
*** prateek has joined #openstack-nova | 04:58 | |
*** links has joined #openstack-nova | 04:58 | |
*** jaosorior_away is now known as jaosorior | 05:00 | |
*** thorst_afk has joined #openstack-nova | 05:01 | |
*** claudiub has joined #openstack-nova | 05:03 | |
*** thorst_afk has quit IRC | 05:05 | |
*** crushil has joined #openstack-nova | 05:06 | |
*** lpetrut has joined #openstack-nova | 05:06 | |
*** fragatina has quit IRC | 05:11 | |
*** claudiub has quit IRC | 05:12 | |
*** udesale has joined #openstack-nova | 05:18 | |
*** thorst_afk has joined #openstack-nova | 05:22 | |
*** crushil has quit IRC | 05:23 | |
*** ratailor has joined #openstack-nova | 05:24 | |
*** thorst_afk has quit IRC | 05:26 | |
*** dikonoor has quit IRC | 05:26 | |
*** salv-orlando has joined #openstack-nova | 05:27 | |
*** dikonoor has joined #openstack-nova | 05:27 | |
*** Sukhdev has quit IRC | 05:28 | |
*** shaohe_feng has quit IRC | 05:34 | |
*** psachin has quit IRC | 05:35 | |
*** shaohe_feng has joined #openstack-nova | 05:35 | |
*** felipemonteiro has quit IRC | 05:37 | |
*** yamamoto_ has quit IRC | 05:39 | |
*** Daisy has quit IRC | 05:39 | |
*** Daisy has joined #openstack-nova | 05:43 | |
*** salv-orlando has quit IRC | 05:44 | |
openstackgerrit | Eli Qiao proposed openstack/nova master: Re-worked ram filter and weights to support overcommit https://review.openstack.org/390984 | 05:45 |
*** rcernin has joined #openstack-nova | 05:46 | |
*** lpetrut has quit IRC | 05:47 | |
*** mamandle has joined #openstack-nova | 05:48 | |
*** kaisers has joined #openstack-nova | 05:50 | |
openstackgerrit | Jay Pipes proposed openstack/nova master: Fix wrong disk size for sparse flat files https://review.openstack.org/396873 | 05:51 |
*** gyee_ has quit IRC | 05:52 | |
openstackgerrit | Luo Gangyi proposed openstack/nova master: convert unicode to string before we connect to rados https://review.openstack.org/465404 | 05:53 |
*** kaisers_ has quit IRC | 05:53 | |
openstackgerrit | Tao Li proposed openstack/nova master: Fix the display of updated_at time when using memcache driver. https://review.openstack.org/414463 | 05:55 |
*** mdnadeem has joined #openstack-nova | 05:56 | |
*** ijw has joined #openstack-nova | 06:05 | |
*** links has quit IRC | 06:08 | |
*** nurdie has quit IRC | 06:10 | |
*** ijw has quit IRC | 06:10 | |
openstackgerrit | christopher.uhler proposed openstack/nova master: Libvirt volume driver for Veritas HyperScale https://review.openstack.org/443951 | 06:13 |
*** Oku_OS-away is now known as Oku_OS | 06:14 | |
openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Support tag instances when boot(3/4) https://review.openstack.org/459593 | 06:19 |
*** thorst_afk has joined #openstack-nova | 06:23 | |
*** links has joined #openstack-nova | 06:23 | |
*** pcaruana has joined #openstack-nova | 06:24 | |
*** yamamoto_ has joined #openstack-nova | 06:26 | |
*** thorst_afk has quit IRC | 06:27 | |
*** p01nt3r75 has quit IRC | 06:29 | |
openstackgerrit | Eli Qiao proposed openstack/nova master: Re-worked ram filter and weights to support overcommit https://review.openstack.org/390984 | 06:30 |
*** Daisy has quit IRC | 06:33 | |
*** Daisy has joined #openstack-nova | 06:34 | |
*** satyar has joined #openstack-nova | 06:34 | |
*** gouthamr has quit IRC | 06:36 | |
*** Daisy has quit IRC | 06:38 | |
*** Daisy has joined #openstack-nova | 06:40 | |
*** rabel_b1 has joined #openstack-nova | 06:42 | |
*** markus_z has joined #openstack-nova | 06:43 | |
*** rabel_b1 has quit IRC | 06:45 | |
*** Daisy has quit IRC | 06:45 | |
*** dimtruck is now known as zz_dimtruck | 06:45 | |
*** rabel_b1 has joined #openstack-nova | 06:45 | |
*** salv-orlando has joined #openstack-nova | 06:45 | |
*** Daisy has joined #openstack-nova | 06:46 | |
*** salv-orlando has quit IRC | 06:50 | |
*** Daisy has quit IRC | 06:50 | |
*** ralonsoh has joined #openstack-nova | 06:50 | |
*** Daisy has joined #openstack-nova | 06:56 | |
*** gszasz has joined #openstack-nova | 06:56 | |
openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Add tags fields for server list detail/show api-ref https://review.openstack.org/465842 | 07:02 |
*** sofie_ has joined #openstack-nova | 07:05 | |
*** ltomasbo|away is now known as ltomasbo | 07:08 | |
*** belmoreira has joined #openstack-nova | 07:11 | |
*** mnestratov has joined #openstack-nova | 07:15 | |
*** karimb has joined #openstack-nova | 07:15 | |
*** links has quit IRC | 07:16 | |
*** rabel_b1 has quit IRC | 07:17 | |
*** rabel_b1 has joined #openstack-nova | 07:18 | |
*** Daisy has quit IRC | 07:18 | |
*** thorst_afk has joined #openstack-nova | 07:24 | |
*** Daisy has joined #openstack-nova | 07:25 | |
*** rabel has joined #openstack-nova | 07:28 | |
*** thorst_afk has quit IRC | 07:28 | |
*** ltomasbo is now known as ltomasbo|away | 07:28 | |
*** rabel_b1 has quit IRC | 07:29 | |
*** adisky_ has quit IRC | 07:29 | |
*** links has joined #openstack-nova | 07:30 | |
*** Daisy has quit IRC | 07:30 | |
*** rabel_b1 has joined #openstack-nova | 07:30 | |
*** rabel has quit IRC | 07:32 | |
*** zz_dimtruck is now known as dimtruck | 07:34 | |
*** Daisy has joined #openstack-nova | 07:35 | |
*** mlakat has joined #openstack-nova | 07:39 | |
openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Add tags fields for server list detail/show api-ref https://review.openstack.org/465842 | 07:40 |
openstackgerrit | jichenjc proposed openstack/nova master: Use plain routes list for os-quota-sets endpoint instead of stevedore https://review.openstack.org/461630 | 07:41 |
*** lpetrut has joined #openstack-nova | 07:41 | |
openstackgerrit | jichenjc proposed openstack/nova master: Use plain routes list for os-quota-classes endpoint instead of stevedore https://review.openstack.org/461641 | 07:42 |
*** cfriesen has quit IRC | 07:42 | |
*** jpena|off is now known as jpena | 07:44 | |
*** dimtruck is now known as zz_dimtruck | 07:44 | |
*** tovin07_ has joined #openstack-nova | 07:46 | |
openstackgerrit | jichenjc proposed openstack/nova master: Use plain routes list for os-services endpoint instead of stevedore https://review.openstack.org/460818 | 07:46 |
*** salv-orlando has joined #openstack-nova | 07:46 | |
openstackgerrit | jichenjc proposed openstack/nova master: Use plain routes list for os-certificates endpoint instead of stevedore https://review.openstack.org/460813 | 07:48 |
openstackgerrit | jichenjc proposed openstack/nova master: Use plain routes list for os-migrations endpoint instead of stevedore https://review.openstack.org/461628 | 07:50 |
*** salv-orlando has quit IRC | 07:52 | |
*** sdatko has quit IRC | 07:53 | |
*** dpawlik has quit IRC | 07:53 | |
*** ijw has joined #openstack-nova | 07:54 | |
*** ijw has quit IRC | 07:58 | |
*** zzzeek has quit IRC | 08:00 | |
*** zzzeek has joined #openstack-nova | 08:00 | |
*** sdatko has joined #openstack-nova | 08:00 | |
*** dpawlik has joined #openstack-nova | 08:01 | |
*** karimb has quit IRC | 08:07 | |
*** adisky_ has joined #openstack-nova | 08:10 | |
*** aarefiev_afk is now known as aarefiev | 08:16 | |
*** lucas-afk is now known as lucasagomes | 08:16 | |
openstackgerrit | Andrey Volkov proposed openstack/nova master: PoC: Select PCI devices with distinct tag values https://review.openstack.org/448008 | 08:20 |
openstackgerrit | sahid proposed openstack/nova master: libvirt: update the logic to configure volume with scsi controller https://review.openstack.org/459741 | 08:24 |
openstackgerrit | sahid proposed openstack/nova master: libvirt: update logic to configure device for scsi controller https://review.openstack.org/459740 | 08:24 |
openstackgerrit | sahid proposed openstack/nova master: libvirt: update LibvirtConfigGuestDeviceAddress to provide XML https://review.openstack.org/459737 | 08:24 |
openstackgerrit | sahid proposed openstack/nova master: libvirt: rearange how scsi controller is defined https://review.openstack.org/459739 | 08:24 |
openstackgerrit | sahid proposed openstack/nova master: libvirt: set full description of the controller used by disk https://review.openstack.org/459738 | 08:24 |
*** thorst_afk has joined #openstack-nova | 08:24 | |
openstackgerrit | Tao Li proposed openstack/nova master: Fix the display of updated_at time when using memcache driver. https://review.openstack.org/414463 | 08:25 |
*** yassine has joined #openstack-nova | 08:26 | |
*** yassine is now known as Guest16367 | 08:26 | |
*** gongysh has joined #openstack-nova | 08:27 | |
*** takashin has quit IRC | 08:28 | |
*** thorst_afk has quit IRC | 08:29 | |
*** takashin has joined #openstack-nova | 08:31 | |
*** ltomasbo|away is now known as ltomasbo | 08:31 | |
lyarwood | bauzas, sfinucan, johnthetubaguy ; https://review.openstack.org/#/c/462348/ - A pretty straight forward rootwrap fix for something I broke a few weeks ago if you have time to review this morning, I might be feeling slightly guilty about this ;) | 08:32 |
* sfinucan looking | 08:32 | |
*** swebster has quit IRC | 08:33 | |
sfinucan | lyarwood: Yup, that looks fine by me. +2 | 08:34 |
*** karimb has joined #openstack-nova | 08:35 | |
*** zz_dimtruck is now known as dimtruck | 08:35 | |
*** swebster has joined #openstack-nova | 08:35 | |
lyarwood | sfinucan: thanks | 08:37 |
*** djohnsto has quit IRC | 08:37 | |
*** djohnsto has joined #openstack-nova | 08:38 | |
*** moshele has joined #openstack-nova | 08:38 | |
*** zhurong has joined #openstack-nova | 08:40 | |
openstackgerrit | jichenjc proposed openstack/nova master: remove unused policy from policy doc https://review.openstack.org/465505 | 08:43 |
*** avolkov has joined #openstack-nova | 08:44 | |
*** dimtruck is now known as zz_dimtruck | 08:45 | |
*** rabel__b1 has joined #openstack-nova | 08:45 | |
*** derekh has joined #openstack-nova | 08:45 | |
*** rabel_b1 has quit IRC | 08:46 | |
*** salv-orlando has joined #openstack-nova | 08:48 | |
*** salv-orlando has quit IRC | 08:53 | |
*** jaosorior is now known as jaosorior_lunch | 08:57 | |
*** gszasz has quit IRC | 08:57 | |
*** gszasz has joined #openstack-nova | 08:58 | |
*** salv-orlando has joined #openstack-nova | 09:00 | |
*** yamahata has quit IRC | 09:02 | |
*** zhurong has quit IRC | 09:05 | |
*** salv-orlando has quit IRC | 09:05 | |
mlakat | Hi Stackers - we have an interesting problem - basically we have a broken connection to an NFS server, which results that os.path.getsize() hangs forever on a mounted file. Do we have any timeout like functionality in OpenStack? Greenthreads seem to be blocked. | 09:08 |
mlakat | this is the bug btw: https://bugs.launchpad.net/nova/+bug/1691131 | 09:09 |
openstack | Launchpad bug 1691131 in OpenStack Compute (nova) "IO stuck causes nova compute agent outage" [Undecided,In progress] - Assigned to Daniel Gonzalez Nothnagel (dgonzalez) | 09:09 |
*** satyar has quit IRC | 09:10 | |
*** ljjjustin has joined #openstack-nova | 09:12 | |
*** nurdie has joined #openstack-nova | 09:15 | |
kashyap | artom: Looking at your patch | 09:15 |
artom | kashyap, thanks :) | 09:16 |
artom | Also, I tried playing around with updating the volume serial number when we do volume-update, but I don't think that's supported in a live domain | 09:17 |
artom | Ie, we can only change it in the persistent XML | 09:17 |
kashyap | artom: Yeah, I saw your note in the LP bug | 09:17 |
kashyap | artom: I was also wondering, if that line in driver.py should be: | 09:17 |
kashyap | flags |= copy and (libvirt.VIR_DOMAIN_BLOCK_REBASE_COPY | libvirt.VIR_DOMAIN_BLOCK_REBASE_COPY_DEV) or 0 | 09:17 |
kashyap | But maybe doing it as two lines is consistent with the rest | 09:18 |
artom | And having the serial number not match the actual volume ID has apparently not been a problem until now, so I just left it | 09:18 |
artom | kashyap, it's still bitwise or'ing all the flags, right? Might as well stay consistent with what's there already | 09:19 |
kashyap | artom: Yes, it is bitwise OR | 09:19 |
kashyap | s/is/is still/ | 09:19 |
*** nurdie has quit IRC | 09:19 | |
openstackgerrit | Yongfeng Du proposed openstack/nova master: Update the file for IPv4-only or IPv6-only network https://review.openstack.org/465891 | 09:21 |
*** esberglu has joined #openstack-nova | 09:22 | |
*** cdent has joined #openstack-nova | 09:24 | |
*** thorst_afk has joined #openstack-nova | 09:25 | |
*** esberglu has quit IRC | 09:26 | |
*** Daisy has quit IRC | 09:27 | |
*** Daisy has joined #openstack-nova | 09:28 | |
*** djohnsto has quit IRC | 09:29 | |
openstackgerrit | Eli Qiao proposed openstack/nova master: Correct force host checking for LM and evacuation. https://review.openstack.org/465895 | 09:29 |
*** udesale__ has joined #openstack-nova | 09:31 | |
*** udesale has quit IRC | 09:31 | |
openstackgerrit | Eli Qiao proposed openstack/nova master: Correct force host checking for LM and evacuation https://review.openstack.org/465895 | 09:32 |
*** Daisy has quit IRC | 09:32 | |
openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix uuid replacement in aggregate notification test https://review.openstack.org/465896 | 09:33 |
*** Shunli has quit IRC | 09:36 | |
*** zz_dimtruck is now known as dimtruck | 09:36 | |
*** djohnsto has joined #openstack-nova | 09:36 | |
openstackgerrit | Jianghua Wang proposed openstack/nova master: Skip policy rules on attach_network for none network allocation https://review.openstack.org/465903 | 09:41 |
*** tbachman has quit IRC | 09:41 | |
*** cNilesh has quit IRC | 09:42 | |
*** karimb has quit IRC | 09:42 | |
openstackgerrit | Sylvain Bauza proposed openstack/nova master: Scheduler returning alternatives per destination https://review.openstack.org/465175 | 09:43 |
openstackgerrit | Sylvain Bauza proposed openstack/nova master: Delete existing allocations when retrying https://review.openstack.org/460433 | 09:43 |
openstackgerrit | Sylvain Bauza proposed openstack/nova master: Don't delete allocation if instance being scheduled https://review.openstack.org/460177 | 09:43 |
openstackgerrit | Sylvain Bauza proposed openstack/nova master: WIP: Allocate instance by the conductor https://review.openstack.org/460455 | 09:43 |
*** thorst_afk has quit IRC | 09:44 | |
*** sambetts|afk is now known as sambetts | 09:44 | |
*** dimtruck is now known as zz_dimtruck | 09:45 | |
kashyap | mdbooth: Noticed the other day, wow, this is merged: https://review.openstack.org/#/c/383859/ ("Fix races with nfs volume mount/umount") | 09:46 |
mdbooth | kashyap: Yeah :) | 09:47 |
* kashyap admires the persistence | 09:47 | |
*** jwcroppe_ has joined #openstack-nova | 09:48 | |
*** tbachman has joined #openstack-nova | 09:49 | |
*** jwcroppe has quit IRC | 09:50 | |
*** tovin07_ has quit IRC | 09:53 | |
*** rabel__b1 has quit IRC | 09:55 | |
* cdent despairs at the need for so much | 09:59 | |
*** ljjjustin has quit IRC | 10:00 | |
*** jamielennox is now known as jamielennox|away | 10:04 | |
*** tuanluong has quit IRC | 10:09 | |
*** abalutoiu has joined #openstack-nova | 10:09 | |
*** pjm6 has joined #openstack-nova | 10:12 | |
pjm6 | good morning | 10:13 |
pjm6 | i have a problem with live migration, basically I put a cpu_mode: custom and cpu_model: SandyBridge | 10:13 |
pjm6 | when i migrate from compute01 to compute02 it works good | 10:13 |
pjm6 | but the other way arround no, because tthe compute02 has more flags | 10:13 |
pjm6 | but by forcing the cpu_model SandyBridge this should be working, no? | 10:13 |
kashyap | pjm6: Do you have the same cpu_{mode,model} config on all Compute nodes | 10:15 |
pjm6 | kashyap, yes I have | 10:16 |
kashyap | BTW, what release are you trying this with? Can't say more without details | 10:16 |
pjm6 | of course, sorry | 10:16 |
*** pewp has quit IRC | 10:16 | |
pjm6 | i'm in Ocata version | 10:16 |
pjm6 | using nova 15.1.2 | 10:17 |
*** pewp has joined #openstack-nova | 10:17 | |
pjm6 | kashyap, this is old but https://blog.codecentric.de/en/2015/03/true-kvm-live-migration-openstack-icehouse-ceph-based-vm-storage/ it seems that I have the problem #2 | 10:20 |
pjm6 | the error says: Unacceptable CPU info: CPU doesn't have compatibility | 10:23 |
*** tbachman has quit IRC | 10:23 | |
kashyap | pjm6: That was in IceHouse | 10:24 |
*** karimb has joined #openstack-nova | 10:25 | |
pjm6 | kashyap, yes I said that was an older version, but the problem seems similar | 10:25 |
pjm6 | for isntance | 10:25 |
pjm6 | the error log: https://pastebin.com/zd35WqqJ | 10:26 |
pjm6 | then I use the virsh cpu-baseline and I get this: https://pastebin.com/AxUx0kYf | 10:27 |
* kashyap clicks | 10:27 | |
kashyap | pjm6: BTW, just a gentle note: this channel is for devel purposes, maybe you're already aware | 10:28 |
*** salv-orlando has joined #openstack-nova | 10:28 | |
kashyap | pjm6: I'm just curious, any reason you don't use 'host-model'? | 10:30 |
kashyap | Which is actually the default, which use the closest available named-CPU model | 10:30 |
pjm6 | kashyap, i didn't use host-model because I though that if i wanted to use SandyBridge the cpu_mode should be custom | 10:31 |
pjm6 | or i can use cpu_mode: host-model and cpu_model: SandyBridge? | 10:31 |
kashyap | pjm6: Yes, that is true _if_ you specifically want the SandyBridge | 10:31 |
kashyap | pjm6: No, you can't do that | 10:31 |
pjm6 | kashyap, the problem is that i have two different cpus, one is Broadwell the other is SandyBridge | 10:32 |
pjm6 | i was trying to "downgrade" the instruction set in order to allow live migration | 10:32 |
pjm6 | but it seems that when i migrate for the newer to the old, he's comparing physical cpu specs from one server to the other, and then fails | 10:33 |
pjm6 | 2 line of the first logs | 10:33 |
*** salv-orlando has quit IRC | 10:33 | |
kashyap | pjm6: Ah-ha, okay, your problem is clearer. If using the lowest-common denominator CPU model between the two hosts, it should 'just work' | 10:34 |
pjm6 | yes, that's what gived to me the virsh cpu-baseline (which had the two CPU configs) | 10:34 |
*** rdo__ has joined #openstack-nova | 10:35 | |
*** zz_dimtruck is now known as dimtruck | 10:36 | |
*** rdo_ has quit IRC | 10:37 | |
*** dpawlik has quit IRC | 10:38 | |
kashyap | pjm6: That should clearly work. Do you have a libvirt debug log with log filters enabled? And please file a Nova bug with details | 10:38 |
*** dpawlik has joined #openstack-nova | 10:38 | |
pjm6 | kashyap, for now i have debug disabled and don't have any filter log | 10:38 |
pjm6 | but i can activate the debug yes | 10:38 |
pjm6 | I also see that the dumpxml are using the correct cpu model | 10:39 |
kashyap | pjm6: BTW, to enable libvirt log filters, this is the way to go: https://kashyapc.fedorapeople.org/virt/openstack/request-nova-libvirt-qemu-debug-logs.txt | 10:40 |
kashyap | pjm6: What are the libvirt / QEMU versions? Just in case if I happened to have an env. I can test it | 10:40 |
kashyap | (To see if could reproduce the behaviour.) | 10:41 |
pjm6 | kashyap, will see that | 10:41 |
pjm6 | libvirt 2.5.0 | qemu 1:2.8 | 10:42 |
kashyap | pjm6: Do you have a libvirt guest XML for a VM started on both hosts? | 10:43 |
kashyap | (Can you post them somewhere, please) | 10:43 |
pjm6 | kashyap, you want a dumpxml of a VM that was created in compute01 and compute02 right/ | 10:44 |
kashyap | pjm6: Yep | 10:44 |
pjm6 | 1 sec please | 10:44 |
*** dimtruck is now known as zz_dimtruck | 10:46 | |
*** cdent has quit IRC | 10:46 | |
*** openstackgerrit has quit IRC | 10:48 | |
*** phuongnh has joined #openstack-nova | 10:48 | |
kashyap | pjm6: Also, one more: please get `virsh capabilities` from both hosts | 10:52 |
pjm6 | dumpxml: https://pastebin.com/xbuA2BFS and https://pastebin.com/NPRAEjbV | 10:54 |
*** nicolasbock has joined #openstack-nova | 10:56 | |
*** jpena is now known as jpena|lunch | 10:57 | |
pjm6 | virsh capabilities: https://pastebin.com/upCBBTu6 and https://pastebin.com/aLDeQipG | 10:57 |
kashyap | Thanks; will look in a few | 10:58 |
pjm6 | well kashyap | 10:58 |
pjm6 | i found a problem! | 10:58 |
pjm6 | compute node02 for some reason is using broadwell | 10:58 |
kashyap | pjm6: ? | 10:58 |
kashyap | Hehe | 10:58 |
pjm6 | well this is strange | 10:59 |
pjm6 | first i create in compute01 | 10:59 |
pjm6 | and then migrate to compute02 | 10:59 |
pjm6 | the was fine | 10:59 |
pjm6 | and the dumpxml was using sandybridge | 10:59 |
kashyap | pjm6: Can you get the real error from /var/log/libvirt/libvirtd.log, too? | 11:03 |
pjm6 | I don't have that file yet | 11:04 |
*** sdague has joined #openstack-nova | 11:06 | |
sfinucan | (My) life would be so much easier if every instance had a NUMA topology associated with it | 11:07 |
* sfinucan just discovered that there's a NUMATopologyLimits object that appears to have broken tests with it | 11:08 | |
pjm6 | kashyap, i already activate as you have | 11:10 |
pjm6 | in the doc | 11:10 |
kashyap | pjm6: For the actual libvirt error, maybe it's redirected to systemd journal, do you see anything in: | 11:11 |
kashyap | journalctl -u libvirtd -l --since=yesterday -p err | 11:11 |
kashyap | s/yesterday/today/ | 11:11 |
pjm6 | yeah i see error | 11:12 |
pjm6 | but idk if is this | 11:12 |
kashyap | pjm6: Can you paste what it is | 11:12 |
kashyap | sfinucan: What is causing grief? | 11:12 |
pjm6 | sure | 11:12 |
kashyap | Oh, you tell it in the following comment | 11:12 |
*** cdent has joined #openstack-nova | 11:13 | |
pjm6 | internal error: unable to execute QEMU command 'query-cpu-definitions': The command query-cpu-definitions has not been found | 11:13 |
sfinucan | sfinucan: People changing things in patches to fix something, but neglecting to mention that they fixed said thing | 11:14 |
sfinucan | Whoops, kashyap ^ | 11:14 |
kashyap | sfinucan: Yeah, I hear you | 11:14 |
*** udesale__ has quit IRC | 11:15 | |
*** ssbljk has joined #openstack-nova | 11:18 | |
*** ssbljk has left #openstack-nova | 11:19 | |
kashyap | pjm6: I think that error comes as part of a series of commands libvirt invocates during some high-level operation like migration. | 11:22 |
*** smatzek has joined #openstack-nova | 11:22 | |
kashyap | But anyway, ensuring your destination Compute node really is also using the custom CPU model should get you going. As you've noticed the problem. | 11:22 |
* kashyap needs to get something to eat | 11:22 | |
*** dikonoo has joined #openstack-nova | 11:23 | |
*** liverpooler has quit IRC | 11:23 | |
pjm6 | yeah for some reason idk why when creating a VM he don't update the XML | 11:23 |
*** smatzek has quit IRC | 11:23 | |
*** smatzek has joined #openstack-nova | 11:24 | |
*** dikonoor has quit IRC | 11:26 | |
*** salv-orlando has joined #openstack-nova | 11:28 | |
*** jaosorior_lunch is now known as jaosorior | 11:29 | |
pjm6 | kashyap, | 11:29 |
pjm6 | it seems it's working now | 11:29 |
pjm6 | but the virsh dumpxml don't show anything related to cpu | 11:29 |
pjm6 | this is normal/ | 11:29 |
*** ijw has joined #openstack-nova | 11:30 | |
*** mamandle has quit IRC | 11:30 | |
pjm6 | forget it appears, with less info | 11:30 |
*** gianpietro has joined #openstack-nova | 11:31 | |
pjm6 | https://pastebin.com/gbC4inS | 11:31 |
pjm6 | i just restared the service | 11:31 |
*** salv-orlando has quit IRC | 11:33 | |
*** ychuang has joined #openstack-nova | 11:34 | |
*** ijw has quit IRC | 11:34 | |
*** prateek has quit IRC | 11:35 | |
*** prateek has joined #openstack-nova | 11:36 | |
*** ratailor has quit IRC | 11:36 | |
*** zz_dimtruck is now known as dimtruck | 11:37 | |
*** gongysh has quit IRC | 11:39 | |
*** phuongnh has quit IRC | 11:39 | |
*** openstackgerrit has joined #openstack-nova | 11:40 | |
openstackgerrit | Daniel Gonzalez Nothnagel proposed openstack/nova master: WIP: Get disk size of instance in separate process https://review.openstack.org/465653 | 11:40 |
*** Kevin_Zheng has quit IRC | 11:40 | |
*** prateek has quit IRC | 11:40 | |
*** thorst_afk has joined #openstack-nova | 11:44 | |
*** dpawlik has quit IRC | 11:44 | |
*** dimtruck is now known as zz_dimtruck | 11:47 | |
*** sree has quit IRC | 11:49 | |
*** gianpietro has quit IRC | 11:50 | |
*** gianpietro has joined #openstack-nova | 11:51 | |
*** ychuang has quit IRC | 11:51 | |
*** sree has joined #openstack-nova | 11:52 | |
*** dpawlik has joined #openstack-nova | 11:55 | |
*** gianpietro has quit IRC | 11:55 | |
*** gianpietro has joined #openstack-nova | 11:55 | |
*** salv-orlando has joined #openstack-nova | 11:57 | |
*** tbachman has joined #openstack-nova | 11:59 | |
*** claudiub has joined #openstack-nova | 12:00 | |
*** MVenesio has joined #openstack-nova | 12:01 | |
*** litao__ has quit IRC | 12:01 | |
*** jmlowe has quit IRC | 12:02 | |
*** jpena|lunch is now known as jpena | 12:03 | |
*** edmondsw has joined #openstack-nova | 12:03 | |
*** yamamoto_ has quit IRC | 12:03 | |
*** catintheroof has joined #openstack-nova | 12:09 | |
*** ayogi has quit IRC | 12:17 | |
*** markvoelker has joined #openstack-nova | 12:23 | |
*** pchavva has joined #openstack-nova | 12:24 | |
*** rfolco has joined #openstack-nova | 12:25 | |
*** mriedem has joined #openstack-nova | 12:28 | |
*** sree has quit IRC | 12:28 | |
*** liverpooler has joined #openstack-nova | 12:29 | |
*** gianpietro has quit IRC | 12:32 | |
openstackgerrit | jichenjc proposed openstack/nova master: remove unused policy from policy doc https://review.openstack.org/465505 | 12:36 |
*** zz_dimtruck is now known as dimtruck | 12:38 | |
*** vladikr has joined #openstack-nova | 12:39 | |
*** salv-orlando has quit IRC | 12:39 | |
*** eharney has joined #openstack-nova | 12:40 | |
*** salv-orlando has joined #openstack-nova | 12:40 | |
*** vks1 has quit IRC | 12:41 | |
*** cdent has quit IRC | 12:41 | |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Factor out duplicated notification sample data (3) https://review.openstack.org/452820 | 12:42 |
*** liverpooler has quit IRC | 12:42 | |
*** liverpooler has joined #openstack-nova | 12:42 | |
*** sofie_ has quit IRC | 12:43 | |
*** gianpietro has joined #openstack-nova | 12:44 | |
*** yamamoto has joined #openstack-nova | 12:44 | |
*** links has quit IRC | 12:45 | |
*** salv-orlando has quit IRC | 12:45 | |
*** Alex______ has quit IRC | 12:45 | |
kashyap | pjm6: Your paste-bin has expired; but good if youve' got it working | 12:47 |
kashyap | mdbooth: Did you notice the above patch? https://review.openstack.org/465653 : | 12:47 |
kashyap | "[...] While getting instance disk information from the libvirt domain xml, it calls os.path.getsize on the disk path. If this path is unresponsive, e.g. because it is located on an overloaded NFS mount, the nova-compute service hangs and becomes unresponsive. [...]" | 12:47 |
kashyap | But they're using a separate process to query for each disk, which could be ... "interesting" if there are, say, 50 disks | 12:48 |
*** gianpietro has quit IRC | 12:49 | |
*** catintheroof has quit IRC | 12:50 | |
*** gianpietro has joined #openstack-nova | 12:52 | |
*** kylek3h has joined #openstack-nova | 12:54 | |
*** prateek has joined #openstack-nova | 12:55 | |
*** prateek has quit IRC | 12:55 | |
*** gouthamr has joined #openstack-nova | 12:58 | |
mlakat | kashyap, yes, but what other methods could you suggest? | 12:59 |
*** karimb has quit IRC | 12:59 | |
kashyap | mlakat: Hmm, I saw your comment on the thread. | 12:59 |
mlakat | I also added some more info to today's meeting: https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting | 13:00 |
kashyap | And, thanks for the link to that eventlet bug, useful to learn about it | 13:00 |
*** mamandle has joined #openstack-nova | 13:00 | |
*** esberglu has joined #openstack-nova | 13:00 | |
*** esberglu has quit IRC | 13:01 | |
*** gianpietro has quit IRC | 13:01 | |
*** esberglu has joined #openstack-nova | 13:01 | |
*** esberglu has quit IRC | 13:01 | |
*** gianpietro has joined #openstack-nova | 13:01 | |
*** ftersin has joined #openstack-nova | 13:03 | |
openstackgerrit | Naichuan Sun proposed openstack/nova master: xenapi: cached images should be cleaned up by time https://review.openstack.org/465954 | 13:05 |
*** abalutoiu_ has joined #openstack-nova | 13:09 | |
*** cdent has joined #openstack-nova | 13:10 | |
*** mdrabe has joined #openstack-nova | 13:11 | |
*** abalutoiu has quit IRC | 13:12 | |
*** baoli has joined #openstack-nova | 13:13 | |
*** dave-mccowan has joined #openstack-nova | 13:14 | |
*** nurdie has joined #openstack-nova | 13:14 | |
*** karimb has joined #openstack-nova | 13:15 | |
*** esberglu has joined #openstack-nova | 13:18 | |
*** esberglu has quit IRC | 13:18 | |
*** esberglu has joined #openstack-nova | 13:18 | |
*** ijw has joined #openstack-nova | 13:18 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove DeviceIsBusy exception https://review.openstack.org/465723 | 13:19 |
*** takashin has left #openstack-nova | 13:20 | |
*** karimb has quit IRC | 13:22 | |
*** nurdie has quit IRC | 13:23 | |
*** ijw has quit IRC | 13:23 | |
*** nurdie has joined #openstack-nova | 13:25 | |
*** takashin has joined #openstack-nova | 13:26 | |
*** yamahata has joined #openstack-nova | 13:27 | |
*** jdillaman has quit IRC | 13:28 | |
*** smatzek has quit IRC | 13:29 | |
*** takashin has quit IRC | 13:32 | |
*** takashin has joined #openstack-nova | 13:32 | |
artom | mriedem, heya, thanks for +W'ing the bottom 2 tagged attach patches :) Question for you in https://review.openstack.org/#/c/376021/49 - I'm not sure what you mean by moving error handling lower down, as I don't see a way to do it consistently without handling at least 1 new exception in the API layer | 13:35 |
openstackgerrit | Naichuan Sun proposed openstack/nova master: xenapi: cached images should be cleaned up by time https://review.openstack.org/465954 | 13:35 |
openstackgerrit | Stephen Finucane proposed openstack/python-novaclient master: Remove various deprecated options https://review.openstack.org/465502 | 13:36 |
*** Yangyapeng has joined #openstack-nova | 13:37 | |
mriedem | artom: for example https://review.openstack.org/#/c/391941/42/nova/compute/api.py@3786 | 13:38 |
mriedem | you're now raising VolumeTaggedAttachToShelvedNotSupported from nova.compute.api.API.attach_volume | 13:38 |
*** takashin has left #openstack-nova | 13:38 | |
*** takashin has joined #openstack-nova | 13:39 | |
mriedem | but not handling that exception in the REST API controller code https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/volumes.py#L337 | 13:39 |
mriedem | not in that patch anyway | 13:39 |
mriedem | and not handling it in the rest api controller code means it would result in a 500 | 13:39 |
openstackgerrit | Edu Alcaniz proposed openstack/nova master: follow up https://review.openstack.org/465975 | 13:39 |
mriedem | i realize you can't get that exception until the microversion because the tag isn't in the schema yet, | 13:40 |
artom | mriedem, oh, you mean move the error handling to the same *patch* | 13:40 |
artom | Not the same layer | 13:40 |
mriedem | but i'd prefer to see the error handling for the rest api done in the same methods that introduce those exceptions | 13:40 |
mriedem | yes | 13:40 |
mriedem | same patch | 13:40 |
artom | OK, sorry | 13:40 |
mriedem | lower layer in the series of changes is what i meant | 13:40 |
artom | OK, makes sense, will do that :) | 13:41 |
*** salv-orlando has joined #openstack-nova | 13:41 | |
*** hshiina has joined #openstack-nova | 13:43 | |
*** baoli_ has joined #openstack-nova | 13:43 | |
mriedem | melwitt: replied on https://review.openstack.org/#/c/465042/ - i think we can just backport, resolve the conflict and note it in the commit message | 13:43 |
mriedem | i don't think we need 2 patches on master for that | 13:44 |
cdent | mriedem: I think edleafe is at pycon, jaypipes in china, and i have to go outside to wait for someone to come rescue my car, so I think bauzas (if he's around) will be your go to guy for today's meeting's scheduler update | 13:44 |
mriedem | cdent: i can run it actually, kid is home sick so i'll be here to do it | 13:45 |
bauzas | yup I'm there | 13:45 |
cdent | i assumed you were running it, mriedem, just the scheduler part, which looks like bauzas has under control | 13:46 |
mriedem | oh i see | 13:46 |
mriedem | ok | 13:46 |
*** salv-orlando has quit IRC | 13:46 | |
*** baoli has quit IRC | 13:46 | |
bauzas | fine by me | 13:47 |
*** Yangyapeng has quit IRC | 13:47 | |
*** lbragstad has joined #openstack-nova | 13:48 | |
*** jamielennox|away is now known as jamielennox | 13:49 | |
*** cdent has quit IRC | 13:49 | |
*** dimtruck is now known as zz_dimtruck | 13:51 | |
*** gianpietro has quit IRC | 13:51 | |
*** moshele has quit IRC | 13:53 | |
*** nurdie has quit IRC | 13:54 | |
*** gianpietro has joined #openstack-nova | 13:55 | |
*** psachin has joined #openstack-nova | 13:56 | |
mriedem | grenade live migration job is basically 100% fail http://logs.openstack.org/42/465042/5/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/6aae0c3/console.html#_2017-05-18_00_39_25_007223 | 13:57 |
mriedem | Failed to restart devstack@g-api.service: Unit devstack@g-api.service not found. | 13:57 |
mriedem | i assume because on master that code is expecting systemd, but started on ocata with screen | 13:57 |
*** gianpietro has quit IRC | 14:00 | |
*** Swami has joined #openstack-nova | 14:00 | |
*** gianpietro has joined #openstack-nova | 14:00 | |
sdague | yeh, though that should have been pinned | 14:00 |
*** awaugama has joined #openstack-nova | 14:01 | |
sdague | mriedem: oh, the live migration scripts are all super custom | 14:01 |
sdague | they are going to need adjustments | 14:01 |
*** mlavalle has joined #openstack-nova | 14:02 | |
*** dikonoo has quit IRC | 14:02 | |
sdague | mriedem: grenade completed fine | 14:02 |
*** dikonoo has joined #openstack-nova | 14:02 | |
sdague | the scripts after that are making assumptions | 14:02 |
*** jmlowe has joined #openstack-nova | 14:03 | |
mriedem | sdague: will dig after the nova meeting | 14:03 |
*** coreywright has quit IRC | 14:06 | |
*** felipemonteiro has joined #openstack-nova | 14:06 | |
*** Yangyapeng has joined #openstack-nova | 14:06 | |
*** hongbin has joined #openstack-nova | 14:08 | |
*** felipemonteiro_ has joined #openstack-nova | 14:09 | |
*** smatzek has joined #openstack-nova | 14:10 | |
*** yamamoto has quit IRC | 14:11 | |
*** mriedem has left #openstack-nova | 14:12 | |
*** mriedem has joined #openstack-nova | 14:12 | |
*** yamamoto has joined #openstack-nova | 14:12 | |
*** felipemonteiro__ has joined #openstack-nova | 14:12 | |
*** felipemonteiro has quit IRC | 14:12 | |
*** krtaylor has quit IRC | 14:13 | |
mriedem | https://bugs.launchpad.net/nova/+bug/1691769 | 14:15 |
openstack | Launchpad bug 1691769 in OpenStack Compute (nova) "gate-grenade-dsvm-neutron-multinode-live-migration-nv fails in pike: "Failed to restart <email address hidden>: Unit <email address hidden> not found."" [Medium,Triaged] | 14:15 |
*** felipemonteiro_ has quit IRC | 14:15 | |
*** yamamoto has quit IRC | 14:16 | |
*** zz_dimtruck is now known as dimtruck | 14:17 | |
*** diablo_rojo has joined #openstack-nova | 14:18 | |
*** mkoderer has joined #openstack-nova | 14:19 | |
*** coreywright has joined #openstack-nova | 14:19 | |
*** gyee has joined #openstack-nova | 14:23 | |
*** cfriesen has joined #openstack-nova | 14:29 | |
*** tbachman has quit IRC | 14:31 | |
*** nurdie has joined #openstack-nova | 14:32 | |
*** yamamoto has joined #openstack-nova | 14:33 | |
*** Kevin_Zheng has joined #openstack-nova | 14:33 | |
*** tbachman has joined #openstack-nova | 14:37 | |
takashin | dansmith: Would you review https://review.openstack.org/#/c/408955/ again? | 14:40 |
*** krtaylor has joined #openstack-nova | 14:42 | |
*** READ10 has joined #openstack-nova | 14:42 | |
*** crushil has joined #openstack-nova | 14:43 | |
*** hongbin has quit IRC | 14:43 | |
*** krtaylor has quit IRC | 14:44 | |
mriedem | takashin: i'm confused, | 14:44 |
mriedem | https://review.openstack.org/#/c/408964/49/nova/api/openstack/api_version_request.py and https://review.openstack.org/#/c/459483/3/nova/api/openstack/api_version_request.py are competing for the same microversion, | 14:44 |
mriedem | but i thought the abort cold migration blueprint dependent on being able to list cold migrations? | 14:45 |
*** mdrabe has quit IRC | 14:45 | |
mriedem | oh nvm | 14:45 |
mriedem | i'm thinking of a different blueprint | 14:45 |
mriedem | this is cold migration with a target host | 14:45 |
*** mdrabe has joined #openstack-nova | 14:45 | |
*** lpetrut has quit IRC | 14:48 | |
*** krtaylor has joined #openstack-nova | 14:48 | |
*** claudiub has quit IRC | 14:48 | |
*** tbachman has quit IRC | 14:49 | |
*** mdrabe has quit IRC | 14:50 | |
takashin | mriedem: As you said, there is no dependency between "cold migration with target host" and "List/show all migration types" blueprint. | 14:51 |
*** karimb has joined #openstack-nova | 14:53 | |
*** jwcroppe_ has quit IRC | 14:53 | |
openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix uuid replacement in aggregate notification test https://review.openstack.org/465896 | 14:55 |
mriedem | sdague: i'm confused that USE_SCREEN=False on the old side of this live migration grenade job http://logs.openstack.org/42/465042/5/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/6aae0c3/logs/old/local_conf.txt.gz | 14:55 |
openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix uuid replacement in aggregate notification test https://review.openstack.org/465896 | 14:55 |
mriedem | i thought screen was still the default in devstack in ocata? | 14:55 |
*** armax has joined #openstack-nova | 14:56 | |
mriedem | i guess it is https://github.com/openstack-dev/devstack/blob/stable/ocata/stackrc#L88 | 14:56 |
*** mdrabe has joined #openstack-nova | 14:56 | |
*** nurdie has quit IRC | 14:57 | |
*** gianpietro has quit IRC | 14:57 | |
sdague | mriedem: the confusion is there are 3 operating modes right now | 14:58 |
sdague | USE_SCREEN=False before a month ago, means direct exec everything in bash and hope you captured the pid correctly to kill it later | 14:59 |
sdague | we've used that in the gate for the last 3 years because screen races | 14:59 |
sdague | and in the non repeat state, it's fine. | 14:59 |
sdague | it is a different home grown init system with different bad trade offs than screen | 15:00 |
*** jaosorior is now known as jaosorior_away | 15:00 | |
*** yingjun has joined #openstack-nova | 15:01 | |
*** psachin has quit IRC | 15:01 | |
dpawlik | Hi.If you have some time pls review my PS https://review.openstack.org/#/c/459753/ | 15:01 |
dpawlik | thanks a lot | 15:02 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Transform aggregate.add_host notification https://review.openstack.org/396210 | 15:02 |
ftersin | mriedem: hi. who could review https://review.openstack.org/#/c/407440/ ? | 15:04 |
*** gianpietro has joined #openstack-nova | 15:04 | |
*** takashin has left #openstack-nova | 15:04 | |
ftersin | mriedem: who of you guys is more familiar with that area? | 15:04 |
mriedem | ftersin: mdbooth | 15:04 |
ftersin | mriedem: thx | 15:05 |
*** gianpiet_ has joined #openstack-nova | 15:05 | |
*** dpawlik has quit IRC | 15:05 | |
ftersin | mdbooth: hi. are you around? | 15:07 |
mdbooth | ftersin: Yep | 15:07 |
*** Oku_OS is now known as Oku_OS-away | 15:07 | |
ftersin | mdbooth: could you look at https://review.openstack.org/#/c/407440/ again? | 15:07 |
mdbooth | Yeah, sure. | 15:08 |
*** Guest16367 has quit IRC | 15:08 | |
mdbooth | So, my issue with that patch has always been the number of changes required in driver.py | 15:08 |
*** yamamoto has quit IRC | 15:08 | |
*** gianpietro has quit IRC | 15:08 | |
*** dpawlik has joined #openstack-nova | 15:08 | |
mdbooth | Which is obvious an artifact of the imagebackend interface being completely borked | 15:08 |
ftersin | mdbooth: i addressed some your comments there. so, it'd be good to get new ones :) | 15:08 |
mdbooth | Somebody should clean that up... ;) | 15:09 |
mdbooth | Anyway, we also agreed at some point to move on with it anyway, so I'll try to look again in that light | 15:09 |
mriedem | sdague: ok, so maybe i didn't ask the right question, | 15:09 |
mriedem | i'm basically wondering what i can key off of to fix this grenade live migration issue | 15:10 |
sdague | mriedem: no | 15:10 |
sdague | you are going to need more complex logic | 15:10 |
openstackgerrit | Chris Friesen proposed openstack/nova master: correctly log port id in neutron api https://review.openstack.org/466003 | 15:10 |
cfriesen | ^ trivial fix to logging | 15:10 |
sdague | mriedem: https://github.com/openstack-dev/devstack/blob/6a62619cbc28625755a76eae08b529438eb4f281/stackrc#L166-L173 | 15:11 |
sdague | that's the gate level work around, and why grenade works | 15:11 |
cfriesen | crap, let me respin that...an extra file snuck in | 15:11 |
*** annegentle has joined #openstack-nova | 15:11 | |
*** jwcroppe has joined #openstack-nova | 15:12 | |
*** claudiub has joined #openstack-nova | 15:12 | |
sdague | mriedem: USE_SCREEN=False is always the case for the gate | 15:12 |
sdague | it's just that we decided to make the transition simpler by making that mean something else | 15:13 |
sdague | so 95% of things just worked | 15:13 |
dpawlik | mriedem: thanks for review. DocImpact-Group: openstack/glance is because someone on this channel told me that docs are related to Openstack Glance. I will create a new PS soon. Thanks! | 15:13 |
mriedem | sdague: ok i see that in devstack-gate localrc_set "$localrc_file" "USE_SCREEN" "False" | 15:13 |
sdague | the grenade cases needed this one extra hit | 15:13 |
sdague | hint | 15:13 |
ftersin | mdbooth: this patch does not bring a lot of changes for driver.py. Moreover, some changed might be separated out of this patch. E.g. see a series started with https://review.openstack.org/#/c/411942 | 15:13 |
sdague | then you are dealing with things that wrote their own service management logic outside of devstack | 15:13 |
sdague | which reacted poorly | 15:14 |
sdague | because their start/stop/restart calls aren't going through devstack code with vars that would have done the right things | 15:14 |
openstackgerrit | Chris Friesen proposed openstack/nova master: correctly log port id in neutron api https://review.openstack.org/466003 | 15:14 |
sdague | so options are: 1) replicate special logic | 15:14 |
sdague | 2) redo them to use devstack functions | 15:15 |
sdague | 3) not care about this one job that's in a tarpit | 15:15 |
sdague | given that general live migration post upgrade is already tested, and multi backend live migration on master is already tested | 15:15 |
sdague | just not the combo of the 2 | 15:15 |
*** aarefiev is now known as aarefiev_afk | 15:15 | |
mriedem | general live migration post upgrade being the grenade multinode job? | 15:16 |
sdague | yes | 15:16 |
sdague | so the missing test coverage right now is post upgrade live migration with ceph / nfs | 15:17 |
mriedem | only ceph actually | 15:17 |
mriedem | the nfs stuff was never fixed | 15:17 |
sdague | ok, so only ceph | 15:17 |
mriedem | what's multibackend live migration? | 15:17 |
sdague | also, note, it's not a vm that predates the migration | 15:17 |
mriedem | the back and forth thing? | 15:17 |
sdague | mriedem: the normal livemigration job | 15:17 |
sdague | that tests local disk, and ceph, and I guess not nfs any more | 15:17 |
mriedem | the back and forth thing was meant to test live migrating from n to n-1 and back | 15:18 |
*** gomarivera_ has joined #openstack-nova | 15:18 | |
sdague | oh, right, this is mixed computes? | 15:18 |
sdague | right so mixed computes with ceph, not tested | 15:18 |
sdague | that's what's currently lost | 15:18 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Transform aggregate.remove_host notification https://review.openstack.org/396211 | 15:19 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: Plumbing for tagged nic attachment https://review.openstack.org/415622 | 15:20 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: Delete nic metadata when detaching interface https://review.openstack.org/435387 | 15:20 |
*** cdent has joined #openstack-nova | 15:20 | |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: Delete disk metadata when detaching volume https://review.openstack.org/435082 | 15:20 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: Plumbing for tagged volume attachment https://review.openstack.org/391941 | 15:20 |
openstackgerrit | Artom Lifshitz proposed openstack/nova master: API support for tagged device attachment https://review.openstack.org/376021 | 15:20 |
mriedem | sdague: i'm confused as to why i don't see live migration tempest tests running in the normal grenade multinode job then http://logs.openstack.org/42/465042/5/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/22cb09f/console.html | 15:21 |
mriedem | oh b/c smoke | 15:21 |
mriedem | yeah, that was another thing the grenade live miration job did, was bypass the smoke test stuff | 15:21 |
mriedem | so the normal grenade multinode job is not running live migration tests since they aren't smoke tests | 15:21 |
*** moshele has joined #openstack-nova | 15:21 | |
sdague | um... really? | 15:21 |
sdague | I thought we added those back in tehre | 15:22 |
*** slaweq has joined #openstack-nova | 15:22 | |
mriedem | yeah it's why we added this https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/devstack-gate.yaml#L1698 | 15:22 |
*** slaweq has quit IRC | 15:22 | |
mriedem | to the grenade live migration job | 15:22 |
sdague | ah | 15:22 |
mriedem | so we don't have mixed compute live migration back and forth from n to n-1 and back testing | 15:23 |
sean-k-mooney | mriedem: QQ what is the hosting requirements(days to keep logs) for the 3rd party ci logs? the systemd change has tripled the log size so we are running out of space. | 15:23 |
*** slaweq has joined #openstack-nova | 15:23 | |
mriedem | sean-k-mooney: no idea | 15:23 |
mriedem | sean-k-mooney: i'd ask in infra probably | 15:23 |
sdague | mriedem: ok, I honestly though we still had basic live migration in multinode grenade, maybe it wasn't reliable enough | 15:24 |
mriedem | sean-k-mooney: https://docs.openstack.org/infra/system-config/third_party.html | 15:24 |
mriedem | Note All test artifacts must be retained for one month. | 15:24 |
sean-k-mooney | mriedem: we were aiming for 90 days before im looking into geting more storage but we might have to reduce for a bit while we get that set up | 15:24 |
mriedem | sean-k-mooney: sounds like 1 month minimum | 15:24 |
sdague | mriedem: ok, so option 4) don't do the special ceph live migration job | 15:24 |
*** hongbin has joined #openstack-nova | 15:24 | |
sdague | and just make that job run live migration grenade tests | 15:24 |
sean-k-mooney | mriedem: cool thats what i taught. we might reduce to 45 days while we see if we can upgrade form 500GB to 1TB of hosting | 15:25 |
sdague | because it's the post_hook that's a bit hacking and a problem | 15:25 |
sean-k-mooney | the nova jobs are eating ~ 13.5GB per day post the systemd change | 15:25 |
mriedem | sdague: the special ceph live migration job being the non-grenade live migration job that runs with ceph too? that one isn't a problem. | 15:26 |
sdague | mriedem: I'm saying it all got too compled | 15:26 |
sdague | there is grenade-multi | 15:26 |
mriedem | gate-grenade-dsvm-neutron-multinode-ubuntu-xenial | 15:27 |
mriedem | yes | 15:27 |
sdague | sorry, let me blast this out | 15:27 |
* mriedem braces for impact | 15:27 | |
sdague | grenade-multi - no ceph, no live migration | 15:27 |
sdague | dsvm-multi - no ceph, live migration | 15:27 |
sdague | live-migration - ceph, live migration (custom post_hook, fixed) | 15:27 |
sdague | grenade-live-migration - ceph, live migration (custom post_hook as above, doesn't work across systemd transition) | 15:28 |
sdague | there is no grenade-multi-full | 15:28 |
sdague | that would give you working live migration testing today | 15:28 |
sdague | the equiv of dsvm-multi | 15:29 |
*** jdillaman has joined #openstack-nova | 15:29 | |
*** gomarivera_ has quit IRC | 15:29 | |
mriedem | yeah, now i'm trying to see what's setting live_migrate_back_and_forth in tempest.conf | 15:30 |
*** winston-d_ has joined #openstack-nova | 15:30 | |
mriedem | because the dsvm-multi job isn't doing it | 15:30 |
mriedem | ah only set in the grenade-dsvm-neutron-multinode-live-migration job | 15:31 |
*** tbachman has joined #openstack-nova | 15:33 | |
mriedem | but i don't actually see that set in tempest.conf in the job, probably because of the hackery in the hooks in nova | 15:33 |
mriedem | for that job | 15:33 |
*** salv-orlando has joined #openstack-nova | 15:33 | |
*** fragatina has joined #openstack-nova | 15:33 | |
*** fragatina has quit IRC | 15:33 | |
mriedem | man i don't even know anymore, i might just give up | 15:34 |
*** fragatina has joined #openstack-nova | 15:34 | |
*** nurdie has joined #openstack-nova | 15:37 | |
*** moshele has quit IRC | 15:37 | |
*** mamandle has quit IRC | 15:44 | |
*** jamesdenton has joined #openstack-nova | 15:45 | |
*** markus_z has quit IRC | 15:45 | |
*** dikonoo has quit IRC | 15:45 | |
*** dikonoor has joined #openstack-nova | 15:45 | |
openstackgerrit | Tao Li proposed openstack/nova master: Fix the display of updated_at time when using memcache driver. https://review.openstack.org/414463 | 15:47 |
*** rcernin has quit IRC | 15:47 | |
mriedem | ildikov: johnthetubaguy: i can't make the nova/cinder meeting today | 15:52 |
mriedem | stvnoyes: ^ | 15:53 |
mriedem | i also haven't had a chance to run through steve's live migration wip changes yet | 15:53 |
ildikov | mriedem: thanks for the heads up | 15:54 |
*** Apoorva has joined #openstack-nova | 15:55 | |
sean-k-mooney | mriedem: were you looking for where in the code the live_migration flag is set in the gate-grenade-dsvm-neutron-multinode-live-migration-nv or just what file it was set in | 15:55 |
mriedem | sean-k-mooney: it's set in local.conf https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/devstack-gate.yaml#L1703 | 15:56 |
mriedem | and should be set in tempest.conf during devstack setup here https://github.com/openstack-dev/devstack/blob/master/lib/tempest#L364 | 15:57 |
kashyap | cfriesen: Looked at your patch this morning, responded on it | 15:57 |
sean-k-mooney | mriedem: ah you found it so i was goint to take a look and see if i could find it | 15:57 |
mriedem | but i'm not actually seeing it set in tempest.conf in that job | 15:57 |
kashyap | cfriesen: It looks fine to me, nice catch. I presume you stumbled on it by testing | 15:57 |
sean-k-mooney | i am | 15:57 |
kashyap | (I mean, in real env) | 15:57 |
*** dpawlik is now known as _dpawlik | 15:57 | |
sean-k-mooney | mriedem: http://logs.openstack.org/60/465160/2/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/c34cb8b/logs/new/tempest_conf.txt.gz | 15:57 |
*** belmoreira has quit IRC | 15:58 | |
mriedem | sean-k-mooney: i don't see live_migrate_back_and_forth in there | 15:58 |
sean-k-mooney | or sorry i taught you were just looking for live_migration = True sorry | 15:58 |
mriedem | it's in the loca.conf though http://logs.openstack.org/42/465042/5/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/6aae0c3/logs/new/local.conf.txt.gz | 15:58 |
mriedem | LIVE_MIGRATE_BACK_AND_FORTH=True | 15:58 |
dansmith | mriedem: omg, is that really a flag? | 16:00 |
*** gszasz has quit IRC | 16:00 | |
mriedem | it is | 16:00 |
dansmith | love. it. | 16:00 |
mriedem | i must be missing something | 16:01 |
cfriesen | kashyap: thanks for the review...we did indeed trip over it | 16:01 |
sean-k-mooney | mriedem: hum maybe it would be better to skip the macro and update the tempest conf via setting [[test-config|/opt/stack/new/tempest/etc/tempest.conf]] ... section in the local.conf | 16:01 |
mriedem | sean-k-mooney: that didn't exist when this was added | 16:01 |
mriedem | and this used to work, i'm pretty sure | 16:01 |
*** karthiks has quit IRC | 16:01 | |
mriedem | i see the devstack flag in the grenade log | 16:01 |
mriedem | http://logs.openstack.org/42/465042/5/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/6aae0c3/logs/grenade.sh.txt.gz#_2017-05-18_00_34_34_956 | 16:01 |
kashyap | cfriesen: Should that be documented the format-domain for libvirt upstream, too? | 16:02 |
dansmith | mriedem: probably unintentional, but just thinking of the potential "me and you and everyone we know" reference | 16:02 |
sean-k-mooney | mriedem: well its only used once in devstack and its pretty clear what its trying to do so that is strange that its not working https://github.com/openstack-dev/devstack/blob/ac042d41222eea83bf360b4db89c4c7bf415457d/lib/tempest#L364 | 16:02 |
mriedem | wtf it should happen right here http://logs.openstack.org/42/465042/5/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/6aae0c3/logs/grenade.sh.txt.gz#_2017-05-18_00_21_27_068 | 16:02 |
kashyap | cfriesen: Okay, currently I see it documented under 'vcpu' -> 'cpuset' there | 16:02 |
*** gomarivera_ has joined #openstack-nova | 16:03 | |
openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: remove related pinning from __init__ of InstanceNUMATopology https://review.openstack.org/408242 | 16:03 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: remove pagesize from __init__ of InstanceNUMATopology https://review.openstack.org/408241 | 16:03 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: remove cpuset_reserved from __init__ of InstanceNUMATopology https://review.openstack.org/466030 | 16:03 |
kashyap | cfriesen: Probably that's good enough: "Each element in that list is either a single CPU number, a range of CPU numbers, or a caret followed by a CPU number to be excluded from a previous range." | 16:03 |
sfinucan | dansmith: Finally got one of those working ^ | 16:03 |
mriedem | sean-k-mooney: i see the problem | 16:03 |
dansmith | sfinucan: cool | 16:03 |
mriedem | https://github.com/openstack-dev/devstack/blob/stable/ocata/lib/tempest | 16:03 |
mriedem | the devstack flag didn't get into devstack stable/ocata | 16:03 |
mriedem | and tempest.conf is configured on the old side in the grenade run | 16:04 |
sfinucan | dansmith: ...however, 'nova.tests.unit.virt.test_hardware.HelperMethodsTestCase.test_dict_numa_topology_to_obj' is failing for the other two and I can't figure out why | 16:04 |
*** aloga has quit IRC | 16:04 | |
*** aloga has joined #openstack-nova | 16:04 | |
sean-k-mooney | mriedem: ah ok that makes sense | 16:04 |
sfinucan | If you have a chance to look at that at some point, could you? | 16:04 |
sfinucan | For some reason, a dict is not behaving like a dict :S | 16:04 |
sfinucan | stupid ovo | 16:05 |
cfriesen | kashyap: yes, I think that's okay. the implicit all-cpus was just in the nova code. | 16:05 |
mriedem | sean-k-mooney: so i just needed to backport the fix to devstack https://review.openstack.org/#/c/466033/ | 16:05 |
mriedem | now having done that, the grenade live migration job is still broken, as discussed with sdague earlier | 16:06 |
mriedem | and i'm not sure i have the energy atm to untangle that mess | 16:06 |
sean-k-mooney | mriedem: yes backport or swap to useing the test-config update to the file which is supported on ocata already https://github.com/openstack-dev/devstack/blob/stable/ocata/stack.sh#L1405 | 16:07 |
kashyap | cfriesen: True. Okay, I'll go cook food, and head to my Dutch class, then :-) | 16:07 |
*** yingjun has quit IRC | 16:07 | |
*** mdnadeem has quit IRC | 16:08 | |
*** jose-phillips has quit IRC | 16:08 | |
sean-k-mooney | mriedem: oh the flag is new. backporting makes sense then | 16:08 |
*** yamamoto has joined #openstack-nova | 16:09 | |
*** jose-phillips has joined #openstack-nova | 16:09 | |
*** burt has joined #openstack-nova | 16:12 | |
sean-k-mooney | sfinucan: how dare you disbarage ovo :P | 16:13 |
sfinucan | sean-k-mooney: I'm sure it's lovely, but I've just burnt through a full day looking at ovo-related issues. I ain't feeling it today ;) | 16:13 |
sean-k-mooney | there a child of nova object you have to treat them kindly but ya they can be a pain. | 16:14 |
*** Yangyapeng has quit IRC | 16:14 | |
*** yamamoto has quit IRC | 16:15 | |
alex_xu | mriedem: sdague sfinucan oomichi there are some nova bug fix in the etherpad from bugsmash https://etherpad.openstack.org/p/OpenStack-Bug-Smash-Pike-Suzhou-Bug-List, appreciate if you can review few of them when you have free time, thanks in advance! | 16:16 |
*** gianpietro has joined #openstack-nova | 16:16 | |
openstackgerrit | Alex Xu proposed openstack/nova master: Correct the error message for query parameter validation https://review.openstack.org/465785 | 16:17 |
*** yingjun has joined #openstack-nova | 16:17 | |
*** catintheroof has joined #openstack-nova | 16:19 | |
dansmith | mriedem: so, I'm getting this on my cells test: http://logs.openstack.org/37/458537/20/check/gate-tempest-dsvm-multinode-live-migration-ubuntu-xenial/ab364da/logs/testr_results.html.gz | 16:19 |
*** gianpiet_ has quit IRC | 16:19 | |
dansmith | mriedem: but I'm failing because of lack of shared storage, which shouldn't be affected by what I'm doing | 16:19 |
dansmith | oh, um, hmm.. | 16:19 |
*** nic has joined #openstack-nova | 16:20 | |
dansmith | I wonder if it's because I'm mucking with the config for compute.. | 16:20 |
*** pewp has quit IRC | 16:20 | |
*** karimb has quit IRC | 16:21 | |
*** karimb has joined #openstack-nova | 16:23 | |
dansmith | mriedem: isn't there some late hooky stuff going on from the nova tree for the live migration job? | 16:23 |
dansmith | sdague: ^ | 16:23 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-manage: Deprecate 'log' commands https://review.openstack.org/453176 | 16:23 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-manage: Deprecate 'host' commands https://review.openstack.org/453175 | 16:23 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-manage: Deprecate 'project', 'account' commands https://review.openstack.org/453174 | 16:23 |
sfinucan | alex_xu: I'll trade you some of yours for some of mine ;) ^^^ | 16:23 |
sfinucan | artom: Addressed your comment in above change, if you're about ^ | 16:24 |
*** nic has quit IRC | 16:24 | |
zzzeek | sdague: what is a "5 byte utf8" character ? | 16:24 |
artom | sfinucan, I'm about | 16:25 |
*** fragatina has quit IRC | 16:25 | |
*** salv-orlando has quit IRC | 16:25 | |
*** salv-orlando has joined #openstack-nova | 16:26 | |
*** salv-orlando has quit IRC | 16:27 | |
*** pewp has joined #openstack-nova | 16:27 | |
*** salv-orl_ has joined #openstack-nova | 16:27 | |
sfinucan | zzzeek: UTF-8 chars can vary in size from 1 to 6 bytes, it appears https://stackoverflow.com/q/5290182/ | 16:28 |
sdague | zzzeek: there was a nova bug on it at some point | 16:28 |
zzzeek | sdague sfinucan OK but unicode itself only needs four bytes | 16:29 |
zzzeek | sdague sfinucan MySQL / Posgresql / anything only goes out to 4 bytes max | 16:29 |
sdague | ok, good to know, it was in the realm of "um... so not able to go there" | 16:29 |
zzzeek | https://en.wikipedia.org/wiki/UTF-8#Description | 16:29 |
zzzeek | OK | 16:29 |
sean-k-mooney | sfinucan: i thinkg its 1-4 bytes if i remember correctly | 16:30 |
zzzeek | sdague: thougth that was weird, b.c. why not refer to "6 byte support" :) | 16:30 |
sdague | I guess I had misunderstood that unicode consortium had crossed that line | 16:30 |
sfinucan | sean-k-mooney: Are you saying Google lied to me? :O | 16:30 |
mdbooth | IIRC UTF-8 characters can be arbitrarily many bytes, because they may combine arbitrarily many codepoints | 16:30 |
*** yingjun has quit IRC | 16:30 | |
zzzeek | mdbooth: the hypotthetical max is 6 bytes | 16:31 |
*** Swami has quit IRC | 16:31 | |
sfinucan | And this is the point where I go chuck a ball about in the sunshine :) | 16:31 |
sean-k-mooney | sfinucan: yes hehe ill double check but when i was working with it in the past i rember it being 4 bytes max for some reason | 16:31 |
openstackgerrit | Dan Smith proposed openstack/nova master: WIP: De-complicate some of the instance delete path https://review.openstack.org/458636 | 16:31 |
openstackgerrit | Dan Smith proposed openstack/nova master: Move to proper target_cell calling convention https://review.openstack.org/458634 | 16:31 |
openstackgerrit | Dan Smith proposed openstack/nova master: Make target_cell() yield a new context https://review.openstack.org/458635 | 16:31 |
openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Debug race in delete-while-booting test https://review.openstack.org/461513 | 16:31 |
mdbooth | zzzeek: I think that's for a single codepoint? | 16:31 |
openstackgerrit | Dan Smith proposed openstack/nova master: DNM: Canary test against multi-cell devstack https://review.openstack.org/458537 | 16:31 |
openstackgerrit | Dan Smith proposed openstack/nova master: Target cell on local delete https://review.openstack.org/464059 | 16:31 |
mdbooth | A character may be multiple codepoints | 16:31 |
zzzeek | mdbooth: now that you have to show me | 16:32 |
zzzeek | :) | 16:32 |
zzzeek | mdbooth: im pretty sure no such voodoo is used in practice | 16:32 |
mdbooth | zzzeek: Like a-umlaut, for eg | 16:32 |
mdbooth | IIRC that's 2 code points, but 1 char | 16:32 |
zzzeek | mdbooth: i mean, in databases | 16:32 |
*** jmlowe has quit IRC | 16:33 | |
mdbooth | Well it's all strings ultimately. | 16:33 |
mdbooth | It boils down to the definition of a character. | 16:33 |
dansmith | please don't let my snoring interrupt this riveting conversation | 16:34 |
zzzeek | mdbooth: OK! well. writing an email here, specifics, things that affect us right now :) | 16:34 |
sdague | zzzeek: I'll update the pg spec with that, it's good info | 16:35 |
*** gjayavelu has joined #openstack-nova | 16:35 | |
*** dikonoor has quit IRC | 16:35 | |
*** lucasagomes is now known as lucas-afk | 16:35 | |
mdbooth | dansmith zzzeek: Heh, yeah, its unlikely to be of interest to us :) It's of interest when you need to display text and, for eg, select characters with a mouse. I read a fantastic article about it recently but I can't find it right now. | 16:39 |
sean-k-mooney | sfinucan: looking at https://tools.ietf.org/html/rfc3629 section 10 "Security Considerations" we are both right.... the utf-8 encounding would support up to 6 bytes but such encounding produce illegal and only up to 4 bytes are vailid | 16:39 |
mdbooth | Questions like: what should pressing backspace do in a text editor are actually really hard to answer. | 16:40 |
*** pcaruana has quit IRC | 16:40 | |
*** pchavva has quit IRC | 16:41 | |
*** smatzek_ has joined #openstack-nova | 16:42 | |
*** burt has quit IRC | 16:44 | |
*** smatzek has quit IRC | 16:46 | |
*** yamahata has quit IRC | 16:46 | |
*** burt has joined #openstack-nova | 16:46 | |
sean-k-mooney | sdague: zzzeek mdbooth if ye just define the field as an NVARCHAR in sqlalchemy and set the coalation type for the table to utf8_bin or utf8_general_ci/utf8_unicode_ci the db should handel it its self | 16:47 |
*** gjayavelu has quit IRC | 16:50 | |
openstackgerrit | Jianghua Wang proposed openstack/nova master: Skip policy rules on attach_network for none network allocation https://review.openstack.org/465903 | 16:50 |
*** karthiks has joined #openstack-nova | 16:54 | |
*** vks1 has joined #openstack-nova | 16:54 | |
openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/nova master: Read Neutron port 'binding_profile' during boot https://review.openstack.org/449257 | 16:55 |
openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/nova master: Add Neutron port capabilities to devspec in request https://review.openstack.org/451777 | 16:55 |
openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/nova master: Format NIC features using os-traits definitions https://review.openstack.org/466051 | 16:55 |
*** salv-orl_ has quit IRC | 16:56 | |
*** salv-orlando has joined #openstack-nova | 16:56 | |
*** pchavva has joined #openstack-nova | 16:57 | |
*** ltomasbo is now known as ltomasbo|away | 16:57 | |
openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/nova master: Read Neutron port 'binding_profile' during boot https://review.openstack.org/449257 | 16:57 |
*** nurdie has quit IRC | 16:57 | |
*** derekh has quit IRC | 16:58 | |
*** fragatina has joined #openstack-nova | 16:59 | |
*** mlakat has quit IRC | 16:59 | |
*** ralonsoh has quit IRC | 17:00 | |
*** Apoorva_ has joined #openstack-nova | 17:00 | |
*** salv-orlando has quit IRC | 17:00 | |
*** pchavva has quit IRC | 17:02 | |
*** pchavva has joined #openstack-nova | 17:02 | |
*** hshiina has quit IRC | 17:03 | |
*** Apoorva has quit IRC | 17:03 | |
*** sambetts is now known as sambetts|afk | 17:03 | |
*** fragatina has quit IRC | 17:04 | |
*** gomarivera_ has quit IRC | 17:05 | |
*** psachin has joined #openstack-nova | 17:06 | |
*** claudiub has quit IRC | 17:07 | |
*** jpena is now known as jpena|off | 17:07 | |
*** jmlowe has joined #openstack-nova | 17:07 | |
*** jose-phillips has quit IRC | 17:08 | |
*** jmlowe has quit IRC | 17:09 | |
*** jmlowe has joined #openstack-nova | 17:09 | |
*** yamamoto has joined #openstack-nova | 17:11 | |
*** jose-phillips has joined #openstack-nova | 17:12 | |
*** nurdie has joined #openstack-nova | 17:14 | |
*** yamamoto has quit IRC | 17:16 | |
*** jmlowe has quit IRC | 17:17 | |
*** jmlowe has joined #openstack-nova | 17:18 | |
*** nurdie has quit IRC | 17:18 | |
*** karthiks has quit IRC | 17:19 | |
*** claudiub has joined #openstack-nova | 17:20 | |
*** fragatina has joined #openstack-nova | 17:22 | |
*** burt has quit IRC | 17:22 | |
*** yamahata has joined #openstack-nova | 17:23 | |
*** salv-orlando has joined #openstack-nova | 17:29 | |
*** nurdie has joined #openstack-nova | 17:29 | |
*** harlowja has joined #openstack-nova | 17:30 | |
*** jmlowe has quit IRC | 17:32 | |
*** jmlowe has joined #openstack-nova | 17:32 | |
*** salv-orlando has quit IRC | 17:34 | |
*** avolkov has quit IRC | 17:37 | |
*** ijw has joined #openstack-nova | 17:38 | |
*** jmlowe has quit IRC | 17:38 | |
*** jmlowe has joined #openstack-nova | 17:38 | |
*** Apoorva_ has quit IRC | 17:39 | |
*** eandersson has joined #openstack-nova | 17:39 | |
*** Apoorva has joined #openstack-nova | 17:39 | |
*** gyee_ has joined #openstack-nova | 17:41 | |
*** jose-phillips has quit IRC | 17:43 | |
*** gyee has quit IRC | 17:43 | |
*** gyee_ has quit IRC | 17:43 | |
*** gyee has joined #openstack-nova | 17:43 | |
*** jose-phillips has joined #openstack-nova | 17:44 | |
*** ijw has quit IRC | 17:51 | |
*** ijw has joined #openstack-nova | 17:51 | |
*** jmlowe has quit IRC | 17:54 | |
*** jmlowe has joined #openstack-nova | 17:54 | |
*** annegentle has quit IRC | 17:55 | |
*** gjayavelu has joined #openstack-nova | 17:55 | |
*** pchavva has quit IRC | 17:56 | |
*** armax has quit IRC | 17:57 | |
*** vks1 has quit IRC | 17:58 | |
*** jmlowe has quit IRC | 17:59 | |
*** jmlowe has joined #openstack-nova | 18:00 | |
*** pchavva has joined #openstack-nova | 18:00 | |
*** jmlowe has quit IRC | 18:00 | |
*** harlowja has quit IRC | 18:01 | |
*** jmlowe has joined #openstack-nova | 18:01 | |
*** Kevin_Zheng has quit IRC | 18:03 | |
*** fragatina has quit IRC | 18:07 | |
*** jmlowe has quit IRC | 18:08 | |
*** fragatina has joined #openstack-nova | 18:08 | |
*** jmlowe has joined #openstack-nova | 18:08 | |
*** jmlowe has quit IRC | 18:10 | |
*** claudiub has quit IRC | 18:11 | |
*** jmlowe has joined #openstack-nova | 18:11 | |
*** yamamoto has joined #openstack-nova | 18:13 | |
*** harlowja has joined #openstack-nova | 18:15 | |
*** ZZelle_ has joined #openstack-nova | 18:15 | |
*** gianpietro has quit IRC | 18:16 | |
openstackgerrit | melanie witt proposed openstack/nova master: Cache database and message queue connection objects https://review.openstack.org/465042 | 18:17 |
*** gianpietro has joined #openstack-nova | 18:18 | |
ZZelle_ | sfinucan, hi | 18:18 |
*** yamamoto has quit IRC | 18:18 | |
*** markvoelker has quit IRC | 18:19 | |
*** markvoelker has joined #openstack-nova | 18:20 | |
*** smatzek_ has quit IRC | 18:23 | |
*** markvoelker has quit IRC | 18:24 | |
*** jose-phillips has quit IRC | 18:25 | |
*** MVenesio has quit IRC | 18:27 | |
*** Sukhdev has joined #openstack-nova | 18:30 | |
*** armax has joined #openstack-nova | 18:32 | |
*** marst has joined #openstack-nova | 18:32 | |
marst | edleafe: are you around? | 18:34 |
*** claudiub has joined #openstack-nova | 18:34 | |
openstackgerrit | christopher.uhler proposed openstack/nova master: Libvirt volume driver for Veritas HyperScale https://review.openstack.org/443951 | 18:36 |
openstackgerrit | Dan Smith proposed openstack/nova master: DNM: Canary test against multi-cell devstack https://review.openstack.org/458537 | 18:37 |
*** gyee has quit IRC | 18:38 | |
*** thorst_afk has quit IRC | 18:43 | |
*** eharney has quit IRC | 18:48 | |
*** mnestratov has quit IRC | 18:48 | |
*** gianpietro has quit IRC | 18:51 | |
*** eharney has joined #openstack-nova | 18:51 | |
cdent | marst: edleafe is probably on a plane on his way to pycon | 18:53 |
*** dillaman has joined #openstack-nova | 18:55 | |
*** harlowja has quit IRC | 18:56 | |
*** karthiks has joined #openstack-nova | 18:58 | |
marst | cdent: thanks! I've just read his blog post | 18:59 |
*** salv-orlando has joined #openstack-nova | 19:04 | |
*** gianpietro has joined #openstack-nova | 19:05 | |
*** READ10 has quit IRC | 19:05 | |
*** salv-orlando has quit IRC | 19:05 | |
*** annegentle has joined #openstack-nova | 19:06 | |
*** harlowja has joined #openstack-nova | 19:07 | |
*** cdent has quit IRC | 19:10 | |
*** nurdie has quit IRC | 19:10 | |
*** karimb has quit IRC | 19:12 | |
*** karimb has joined #openstack-nova | 19:13 | |
*** crushil has quit IRC | 19:14 | |
*** yamamoto has joined #openstack-nova | 19:15 | |
openstackgerrit | Sean Dague proposed openstack/nova master: see if we can use pg wheels with no bindep https://review.openstack.org/466081 | 19:15 |
*** nurdie has joined #openstack-nova | 19:15 | |
*** MVenesio has joined #openstack-nova | 19:19 | |
*** jose-phillips has joined #openstack-nova | 19:19 | |
*** adisky_ has quit IRC | 19:19 | |
*** MVenesio has quit IRC | 19:19 | |
*** yamamoto has quit IRC | 19:21 | |
*** MVenesio has joined #openstack-nova | 19:22 | |
*** jose-phillips has quit IRC | 19:22 | |
*** jose-phillips has joined #openstack-nova | 19:24 | |
mriedem | dansmith: did you figure out http://logs.openstack.org/37/458537/20/check/gate-tempest-dsvm-multinode-live-migration-ubuntu-xenial/ab364da/logs/testr_results.html.gz ? | 19:26 |
*** MVenesio has quit IRC | 19:30 | |
*** dave-mccowan has quit IRC | 19:31 | |
*** krtaylor has quit IRC | 19:33 | |
*** crushil has joined #openstack-nova | 19:35 | |
*** Sukhdev has quit IRC | 19:35 | |
*** krtaylor has joined #openstack-nova | 19:37 | |
*** gouthamr has quit IRC | 19:37 | |
*** krtaylor has quit IRC | 19:40 | |
*** krtaylor has joined #openstack-nova | 19:41 | |
openstackgerrit | Cedric Brandily proposed openstack/nova master: Correct _ensure_console_log_for_instance implementation https://review.openstack.org/466088 | 19:47 |
mriedem | this seems wrong http://logs.openstack.org/42/465042/6/check/gate-nova-tox-functional-py35-ubuntu-xenial/9829a45/console.html#_2017-05-18_18_43_03_020297 | 19:48 |
mriedem | b"KeyError: 'allocations'" | 19:49 |
*** liverpooler has quit IRC | 19:49 | |
dansmith | mriedem: I found the hook yeah, still working on a fix | 19:49 |
mriedem | ok, | 19:50 |
mriedem | i'm working on disabling the grenade live migration job from running on pike until we fix it, or queens | 19:50 |
mriedem | since right now it's just burning nodes | 19:50 |
*** itlinux has joined #openstack-nova | 19:51 | |
mriedem | this is very weird - http://logs.openstack.org/42/465042/6/check/gate-nova-tox-functional-py35-ubuntu-xenial/9829a45/console.html#_2017-05-18_18_43_03_020297 it's like in the middle of a functional test, we got a different response from placement | 19:51 |
mriedem | like it's a GET / response | 19:52 |
mriedem | but was expecting GET /resource_providers/d41180aa-047e-4bc3-b0f3-11532683400d/allocations | 19:52 |
openstackgerrit | Cedric Brandily proposed openstack/nova master: Correct _ensure_console_log_for_instance implementation https://review.openstack.org/466088 | 19:54 |
*** gouthamr has joined #openstack-nova | 19:56 | |
*** gouthamr has quit IRC | 19:57 | |
melwitt | mriedem: yeah, I didn't know what to make of that | 19:58 |
*** ijw has quit IRC | 20:00 | |
*** mnestratov has joined #openstack-nova | 20:02 | |
*** harlowja has quit IRC | 20:03 | |
mriedem | could have something to do with running those services under eventlet in our functional tests? | 20:03 |
*** eharney has quit IRC | 20:05 | |
*** annegentle has quit IRC | 20:07 | |
*** pchavva has quit IRC | 20:09 | |
*** kornica has joined #openstack-nova | 20:09 | |
*** slaweq has quit IRC | 20:11 | |
*** kornicameister has quit IRC | 20:11 | |
*** slaweq has joined #openstack-nova | 20:11 | |
*** awaugama has quit IRC | 20:11 | |
*** itlinux has quit IRC | 20:12 | |
*** annegentle has joined #openstack-nova | 20:13 | |
*** ijw has joined #openstack-nova | 20:16 | |
*** yamamoto has joined #openstack-nova | 20:17 | |
*** esberglu has quit IRC | 20:18 | |
efried | Where does the hashy thing at the end of a release note .yaml's basename come from? | 20:20 |
openstackgerrit | Dan Smith proposed openstack/nova master: DNM: Canary test against multi-cell devstack https://review.openstack.org/458537 | 20:21 |
artom | efried, the 'reno' command does it | 20:21 |
*** yamamoto has quit IRC | 20:21 | |
efried | artom Is there a doc somewhere that describes the release note process? | 20:23 |
efried | (Cause your answer just raised bigger questions ;-) | 20:23 |
artom | efried, not that I know of, but if you have a devstack env somewhere reno should be in there | 20:23 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Skip ceph in grenade live migration job due to restart failures https://review.openstack.org/466097 | 20:23 |
mriedem | sdague: i think i've found a poor man's compromise ^ | 20:23 |
artom | efried, in which case reno --help will help :) | 20:23 |
*** cdent has joined #openstack-nova | 20:23 | |
*** winston-d_ has quit IRC | 20:23 | |
mriedem | efried: artom: https://docs.openstack.org/developer/nova/releasenotes.html | 20:24 |
efried | artom Yeah, found reno --help; it kinda assumes you know what youre doing already. | 20:24 |
efried | Okay, thank you artom | 20:24 |
mriedem | the hash is auto-generated | 20:24 |
mriedem | tox -e venv -- reno new <your reno name> | 20:24 |
mriedem | dhellmann said he's found that sometimes people copy renos and re-use the same hash which messes things up | 20:25 |
efried | I guessed that would be a no-no | 20:25 |
efried | Thanks mriedem & artom | 20:25 |
mriedem | yeah, plus reviewers wouldn't be able to eyeball that | 20:25 |
mriedem | seems the reno docs build itself should fail if that happens | 20:25 |
*** pchavva has joined #openstack-nova | 20:27 | |
*** gouthamr has joined #openstack-nova | 20:29 | |
*** harlowja has joined #openstack-nova | 20:30 | |
*** esberglu has joined #openstack-nova | 20:30 | |
*** belmoreira has joined #openstack-nova | 20:32 | |
*** krtaylor has quit IRC | 20:34 | |
*** pchavva has quit IRC | 20:36 | |
*** belmoreira has quit IRC | 20:37 | |
*** itlinux has joined #openstack-nova | 20:39 | |
*** gianpietro has quit IRC | 20:39 | |
*** gianpiet_ has joined #openstack-nova | 20:39 | |
*** harlowja has quit IRC | 20:39 | |
*** thorst_afk has joined #openstack-nova | 20:39 | |
*** thorst_afk has quit IRC | 20:40 | |
*** gianpiet_ has quit IRC | 20:45 | |
*** salv-orlando has joined #openstack-nova | 20:45 | |
*** jbernard has quit IRC | 20:47 | |
*** ZZelle_ has quit IRC | 20:49 | |
*** thorst_afk has joined #openstack-nova | 20:50 | |
*** Sukhdev has joined #openstack-nova | 20:50 | |
*** jbernard has joined #openstack-nova | 20:50 | |
*** hamzy has quit IRC | 20:52 | |
*** xyang1 has joined #openstack-nova | 20:54 | |
*** dave-mccowan has joined #openstack-nova | 20:57 | |
*** jmlowe has quit IRC | 20:57 | |
*** jbernard has quit IRC | 20:59 | |
*** cdent has quit IRC | 21:01 | |
*** jbernard has joined #openstack-nova | 21:02 | |
*** gianpietro has joined #openstack-nova | 21:02 | |
*** gianpietro has quit IRC | 21:02 | |
*** sdague has quit IRC | 21:06 | |
*** gianpietro has joined #openstack-nova | 21:08 | |
*** armax has quit IRC | 21:09 | |
*** baoli_ has quit IRC | 21:12 | |
*** jamesdenton has quit IRC | 21:12 | |
*** jhesketh_ has joined #openstack-nova | 21:14 | |
*** karimb has quit IRC | 21:16 | |
*** NightKhaos_ has joined #openstack-nova | 21:17 | |
*** yamamoto has joined #openstack-nova | 21:18 | |
*** jhesketh has quit IRC | 21:18 | |
*** NightKhaos has quit IRC | 21:18 | |
*** swamireddy has quit IRC | 21:18 | |
*** knikolla has quit IRC | 21:18 | |
*** NightKhaos_ is now known as NightKhaos | 21:18 | |
*** rfolco has quit IRC | 21:18 | |
*** yamamoto has quit IRC | 21:22 | |
*** dimtruck is now known as zz_dimtruck | 21:24 | |
openstackgerrit | Alex Szarka proposed openstack/nova master: Add method for verify multiple notifications https://review.openstack.org/465526 | 21:25 |
openstackgerrit | Alex Szarka proposed openstack/nova master: [WIP] Transform instance.exists notification https://review.openstack.org/403660 | 21:25 |
*** ijw has quit IRC | 21:29 | |
*** ijw has joined #openstack-nova | 21:30 | |
*** crushil_ has joined #openstack-nova | 21:30 | |
openstackgerrit | Monty Taylor proposed openstack/nova master: WIP Show results of nested parameters in api-ref https://review.openstack.org/464277 | 21:33 |
openstackgerrit | Monty Taylor proposed openstack/nova master: WIP Add description of server addresses objects https://review.openstack.org/464331 | 21:33 |
*** gjayavelu has quit IRC | 21:34 | |
*** ijw has quit IRC | 21:35 | |
*** edmondsw_ has joined #openstack-nova | 21:36 | |
*** gjayavelu has joined #openstack-nova | 21:36 | |
*** baoli has joined #openstack-nova | 21:37 | |
*** gouthamr has quit IRC | 21:37 | |
*** edmondsw has quit IRC | 21:38 | |
cfriesen | mriedem: you around? | 21:39 |
*** fragatin_ has joined #openstack-nova | 21:39 | |
mriedem | sort of | 21:40 |
cfriesen | mriedem: got a weird scenario...upgrades are in progress, one compute node hasn't been upgraded. It gets rebooted without being upgraded and hits exception.ServiceTooOld | 21:40 |
cfriesen | mriedem: are we just expecting people to upgrade in that case? | 21:41 |
openstackgerrit | Alex Szarka proposed openstack/nova master: Add method for verify multiple notifications https://review.openstack.org/465526 | 21:41 |
*** baoli has quit IRC | 21:42 | |
cfriesen | mriedem: actually, thinking about it a bit I'm not clear why the version of that compute node isn't being seen in the DB | 21:42 |
openstackgerrit | Alex Szarka proposed openstack/nova master: [WIP] Transform instance.exists notification https://review.openstack.org/403660 | 21:42 |
*** fragatina has quit IRC | 21:42 | |
cfriesen | mriedem: maybe the service is still being forced down by our management software, causing its version to be ignored by service_get_minimum_version() | 21:44 |
*** fragatin_ has quit IRC | 21:45 | |
mriedem | that's a dansmith question | 21:45 |
*** fragatina has joined #openstack-nova | 21:46 | |
mriedem | cfriesen: it's blowing up here right? https://github.com/openstack/nova/blob/master/nova/objects/service.py#L307 | 21:46 |
cfriesen | mriedem: yep. | 21:47 |
*** catintheroof has quit IRC | 21:47 | |
cfriesen | down in the sql code we ignore the version of "forced-down" services, so maybe that's it | 21:47 |
*** thorst_afk has quit IRC | 21:47 | |
mriedem | right this query https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L490 | 21:49 |
*** tbachman has quit IRC | 21:49 | |
mriedem | so all of your other computes are newton, and you've got a mitaka forced-down compute, and you're restarting it and it's blowing up because that mitaka compute is older than the minimum for all of the other computes, which is newton | 21:50 |
mriedem | so upgrade that compute to newton | 21:51 |
cfriesen | mriedem: yeah, I'm not certain it's forced down but it's the only thing that makes sense | 21:51 |
mriedem | do you have the service record from the db? | 21:52 |
*** slaweq has quit IRC | 21:52 | |
*** annegentle has quit IRC | 21:52 | |
cfriesen | mriedem: could probably track it down. would indeed be able to tell from that. I'm just reviewing a proposed change, wasn't actually my issue. one of the guys wants to bypass the check and I'm not a big fan. | 21:53 |
*** haplo37 has quit IRC | 21:53 | |
*** crushil_ has quit IRC | 21:53 | |
mriedem | i mean you could PUT /os-services/force-down and mark that service as forced_down=False, which would change the min service version check to say mitaka is the oldest, but that might give you other issues if other things are running thinking that newton is the minimum | 21:53 |
cfriesen | we're not using auto-rpc versioning (yet, at least) so during an uprade I think it'd be okay as long as we haven't set the rpc version to newton yet. will have to go get more info I think. | 21:54 |
*** mdrabe has quit IRC | 21:55 | |
*** armax has joined #openstack-nova | 21:56 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Skip ceph in grenade live migration job due to restart failures https://review.openstack.org/466097 | 21:56 |
*** nurdie has quit IRC | 21:56 | |
*** nic has joined #openstack-nova | 21:57 | |
*** claudiub has quit IRC | 21:59 | |
mriedem | cfriesen: did you happen to try this out in your live migration recreate with the 16 ports and network-changed event issues? https://review.openstack.org/#/c/465787/ | 22:00 |
mriedem | before i go too far down that rabbit hole (probably next week at this rate) it'd be good to know if it helps at all | 22:00 |
*** xyang1 has quit IRC | 22:01 | |
*** baoli has joined #openstack-nova | 22:01 | |
*** nic has quit IRC | 22:02 | |
*** haplo37 has joined #openstack-nova | 22:02 | |
*** efried has quit IRC | 22:02 | |
*** itlinux has quit IRC | 22:03 | |
*** itlinux has joined #openstack-nova | 22:04 | |
*** edmondsw_ has quit IRC | 22:05 | |
*** ijw has joined #openstack-nova | 22:05 | |
*** edmondsw has joined #openstack-nova | 22:06 | |
*** djohnsto has quit IRC | 22:06 | |
*** slaweq has joined #openstack-nova | 22:06 | |
*** edmondsw_ has joined #openstack-nova | 22:07 | |
cfriesen | mriedem: related to the other issue, it turns out that if you force the "old" nova-compute down, then try to remove the "force-down" it will fail because Service.save() checks the minimum version | 22:08 |
cfriesen | mriedem: haven't tried that patch yet, but I plan to. | 22:08 |
mriedem | oh that's a fun chicken and egg | 22:08 |
cfriesen | isn't it just. :) | 22:08 |
*** Jeffrey4l_ has quit IRC | 22:08 | |
*** edmondsw has quit IRC | 22:10 | |
*** edmondsw_ has quit IRC | 22:11 | |
*** crushil_ has joined #openstack-nova | 22:11 | |
*** crushil_ has quit IRC | 22:12 | |
*** kylek3h has quit IRC | 22:13 | |
*** Jeffrey4l has joined #openstack-nova | 22:13 | |
mriedem | well, seems like a bug to report. we could bypass the service min version check on save() i'd think, but i'm not sure what kind of weird situations that puts us in | 22:14 |
mriedem | again, like if you've already restarted control services that are caching what they think is the min service version (newton) and all of a sudden you have a running mitaka compute service | 22:15 |
cfriesen | mriedem: just for fun, the original scenario was that we were trying to take down the compute node to upgrade it, so we marked nova-compute as forced-down, then we tried to down the neutron agent and hit an exception, so we tried to roll back and enable nova-compute and hit exception.ServiceTooOld | 22:15 |
*** slaweq has quit IRC | 22:15 | |
cfriesen | not sure there's a good solution here. | 22:16 |
mriedem | cfriesen: why not just disable the service? | 22:16 |
*** efried has joined #openstack-nova | 22:16 | |
mriedem | PUT /os-services/disable-log-reason | 22:16 |
mriedem | that should take that nova-compute out of scheduling | 22:16 |
cfriesen | mriedem: that's a good point...usually we only force down on a fault. will follow up. | 22:17 |
*** yamamoto_ has joined #openstack-nova | 22:19 | |
*** Sukhdev has quit IRC | 22:20 | |
mriedem | honestly i'm not sure what the use case is for forced-down which is different from disabling the service | 22:21 |
*** tbachman has joined #openstack-nova | 22:21 | |
mriedem | i think it has to do with external tooling forcing the service to be 'down' rather than waiting for the service group api check to do it | 22:21 |
mriedem | we could document that better in the api ref | 22:22 |
mriedem | like, what does doing this actually mean, what impacts / side effects does it have, etc | 22:22 |
*** gyee has joined #openstack-nova | 22:22 | |
mriedem | cfriesen: ^ probably another bug to open for lack of clear docs on these | 22:22 |
mriedem | https://specs.openstack.org/openstack/nova-specs/specs/liberty/implemented/mark-host-down.html was the spec | 22:22 |
mriedem | oh i see, the idea is external service finds out the host is about to die, forces it down and then evacuates the instances off that host | 22:23 |
mriedem | since evacuate checks this flag | 22:23 |
mriedem | we should state in the api-ref that forced-down is not something for routine upgrade/maintenance | 22:24 |
*** yamamoto_ has quit IRC | 22:24 | |
mriedem | as documented here https://docs.openstack.org/ops-guide/ops-maintenance-compute.html#planned-maintenance | 22:24 |
*** diablo_rojo has quit IRC | 22:27 | |
*** salv-orlando has quit IRC | 22:29 | |
mriedem | cfriesen: https://bugs.launchpad.net/nova/+bug/1691871 | 22:30 |
openstack | Launchpad bug 1691871 in OpenStack Compute (nova) "forced-down vs service disable is not documented well in the compute API reference" [Medium,Confirmed] | 22:30 |
*** gjayavelu has quit IRC | 22:30 | |
*** salv-orlando has joined #openstack-nova | 22:34 | |
*** Sukhdev has joined #openstack-nova | 22:36 | |
cfriesen | mriedem: it's actually kind of ironic (always feels funny using that word around openstack) since we were one of the groups pushing for the introduction of the "force-down" feature. now looks like someone's misusing it. | 22:37 |
*** salv-orlando has quit IRC | 22:38 | |
*** Yangyapeng has joined #openstack-nova | 22:39 | |
cfriesen | mriedem: in our environment we have something else that monitors compute node health, so it'll detect the loss of a compute node way earlier than nova will, hence the proper use of force-down | 22:39 |
cfriesen | agree with your bug though | 22:40 |
*** itlinux has quit IRC | 22:44 | |
*** thorst_afk has joined #openstack-nova | 22:48 | |
*** mriedem1 has joined #openstack-nova | 22:53 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Skip ceph in grenade live migration job due to restart failures https://review.openstack.org/466097 | 22:55 |
mriedem1 | melwitt: dansmith: ^ "fixes" the grenade live migration job | 22:55 |
*** mriedem has quit IRC | 22:55 | |
mriedem1 | we lose ceph grenade + live migration until the bug is fixed or we get to queens, | 22:55 |
mriedem1 | but we at least get back testing for mixed-level compute upgrade testing | 22:55 |
mriedem1 | *live migration testing i mean | 22:56 |
mriedem1 | on mixed computes | 22:56 |
mriedem1 | regular multinode grenade doesn't test live migration | 22:56 |
mriedem1 | ps2 had a clean run, ps3 just removed a debug echo statement | 22:56 |
*** zz_dimtruck is now known as dimtruck | 23:00 | |
*** gouthamr has joined #openstack-nova | 23:01 | |
*** Apoorva_ has joined #openstack-nova | 23:02 | |
*** Apoorva has quit IRC | 23:04 | |
*** thorst_afk has quit IRC | 23:06 | |
*** mriedem1 is now known as mriedem | 23:09 | |
*** armax has quit IRC | 23:13 | |
*** gouthamr has quit IRC | 23:13 | |
*** sree has joined #openstack-nova | 23:18 | |
*** yamamoto_ has joined #openstack-nova | 23:20 | |
*** gjayavelu has joined #openstack-nova | 23:21 | |
*** salv-orlando has joined #openstack-nova | 23:22 | |
*** armax has joined #openstack-nova | 23:22 | |
*** gjayavelu has quit IRC | 23:22 | |
*** baoli has quit IRC | 23:23 | |
*** sree has quit IRC | 23:23 | |
melwitt | mriedem: +2 | 23:24 |
melwitt | I'm interested in fixing the bug but I don't understand your comment about using common functions like in https://github.com/openstack-dev/devstack/commit/c006bbdeb26df2c60f43d222bdf918f9e24d551f | 23:24 |
*** yamamoto_ has quit IRC | 23:26 | |
*** mnestratov has quit IRC | 23:26 | |
*** salv-orlando has quit IRC | 23:26 | |
*** gjayavelu has joined #openstack-nova | 23:28 | |
mriedem | sdague said there were some common process start/stop/restart functions in devstack and we could overhaul these scripts to use those, | 23:28 |
mriedem | which would hide the screen vs systemd stuff i think | 23:28 |
melwitt | ah, okay | 23:28 |
mriedem | but if those are available, i'm not sure why these live migration scripts weren't using them in the first place | 23:28 |
mriedem | so i assume there was a good reason those functions from devstack wouldn't work in the first place | 23:28 |
mriedem | but without tdurakov anymore it's hard to say | 23:28 |
melwitt | yeah. okay, I shall endeavor to dig into that a bit. because on the surface, I was just thinking to do something dumb like "if ocata, then use old way" | 23:29 |
melwitt | if it's even possible to check "if ocata" in that script | 23:29 |
*** salv-orlando has joined #openstack-nova | 23:31 | |
*** salv-orlando has quit IRC | 23:31 | |
*** ijw has quit IRC | 23:35 | |
mriedem | the branch is available, the devstack-gate vars are in there | 23:35 |
mriedem | but yeah i thought similar, basically we'd need to revert a patch that removed the old functions it was using | 23:35 |
mriedem | and then put in branching logic for doing old vs new things | 23:35 |
mriedem | did you get the quotas changes done that you were talking about? | 23:36 |
mriedem | for dict something or other | 23:36 |
*** yassine has joined #openstack-nova | 23:36 | |
*** yassine is now known as Guest6334 | 23:37 | |
*** esberglu has quit IRC | 23:37 | |
*** Apoorva_ has quit IRC | 23:39 | |
*** Apoorva has joined #openstack-nova | 23:40 | |
*** mlavalle has quit IRC | 23:40 | |
*** Fdaisuke_ has joined #openstack-nova | 23:43 | |
*** Fdaisuke has quit IRC | 23:43 | |
*** jmlowe has joined #openstack-nova | 23:44 | |
melwitt | mriedem: still working through it now. ran into some snags (of course) on the bottom patch. now it should be mechanical changes going through the stack. so I hope to push it up in an hour or so | 23:51 |
melwitt | this series has been such a pain | 23:52 |
mriedem | ok. i decided that between two evils on my todo list, doing forum session recaps was easier for now. :) | 23:54 |
mriedem | and i can continue to say i'm not reviewing your series becaues i'm waiting for you to update it | 23:54 |
melwitt | yup, that you can. | 23:55 |
*** ijw has joined #openstack-nova | 23:56 | |
*** ijw has quit IRC | 23:56 | |
*** ijw has joined #openstack-nova | 23:57 | |
*** ijw has quit IRC | 23:58 | |
*** felipemonteiro__ has quit IRC | 23:59 | |
*** ijw has joined #openstack-nova | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!