Tuesday, 2019-01-15

*** yamamoto has quit IRC00:03
*** yamamoto has joined #openstack-neutron00:05
*** markvoelker has quit IRC00:06
*** mriedem_away has quit IRC00:07
*** yamamoto has quit IRC00:07
*** yamamoto has joined #openstack-neutron00:11
*** yamamoto has quit IRC00:13
*** yamamoto has joined #openstack-neutron00:14
*** yamamoto has quit IRC00:16
*** wolverineav has quit IRC00:18
*** macza has quit IRC00:18
*** yamamoto has joined #openstack-neutron00:27
*** yamamoto has quit IRC00:31
*** wolverineav has joined #openstack-neutron00:35
*** wolverineav has quit IRC00:48
*** wolverineav has joined #openstack-neutron00:50
*** mattw4 has quit IRC00:54
*** wolverineav has quit IRC00:55
*** goldyfruit has joined #openstack-neutron01:10
*** yamamoto has joined #openstack-neutron01:15
*** yamamoto has quit IRC01:21
*** rkukura has quit IRC01:23
*** fragatina has quit IRC01:34
*** markvoelker has joined #openstack-neutron01:35
*** wolverineav has joined #openstack-neutron01:35
*** wolverin_ has joined #openstack-neutron01:38
*** wolverineav has quit IRC01:39
*** markvoelker has quit IRC01:40
*** wolverin_ has quit IRC01:43
*** sapd1 has quit IRC01:50
*** goldyfruit has quit IRC01:51
*** goldyfruit has joined #openstack-neutron01:52
*** hongbin has joined #openstack-neutron01:57
*** hongbin has quit IRC01:59
*** Dinesh_Bhor has joined #openstack-neutron02:03
*** Dinesh_Bhor has quit IRC02:03
*** dconde has quit IRC02:04
*** hongbin has joined #openstack-neutron02:04
*** macza has joined #openstack-neutron02:06
*** Dinesh_Bhor has joined #openstack-neutron02:09
*** macza has quit IRC02:11
*** goldyfruit has quit IRC02:20
*** goldyfruit has joined #openstack-neutron02:20
*** sapd1_x has joined #openstack-neutron02:21
*** jangutter has joined #openstack-neutron02:22
*** jangutter has quit IRC02:27
*** yamamoto has joined #openstack-neutron02:32
*** yamamoto has quit IRC02:37
*** njohnston has quit IRC02:40
*** aojea has joined #openstack-neutron02:43
*** iyamahat_ has quit IRC02:44
*** yamahata has quit IRC02:45
*** aojea has quit IRC02:47
*** wolverineav has joined #openstack-neutron03:27
*** yamamoto has joined #openstack-neutron03:30
*** wolverineav has quit IRC03:32
*** Dinesh_Bhor has quit IRC03:38
*** Dinesh_Bhor has joined #openstack-neutron03:45
*** abaindur has quit IRC03:50
*** Dinesh_Bhor has quit IRC03:55
*** goldyfruit has quit IRC04:04
*** igordc has quit IRC04:05
*** dave-mccowan has quit IRC04:11
*** lajoskatona has joined #openstack-neutron04:15
*** hongbin has quit IRC04:25
*** yamamoto has quit IRC04:34
*** fragatina has joined #openstack-neutron04:45
*** fragatina has quit IRC04:47
*** fragatina has joined #openstack-neutron04:48
*** Dinesh_Bhor has joined #openstack-neutron04:49
*** fragatina has quit IRC04:51
*** wolverineav has joined #openstack-neutron04:51
*** fragatina has joined #openstack-neutron04:51
*** fragatina has quit IRC04:54
*** _fragatina_ has joined #openstack-neutron04:54
*** _fragatina_ has quit IRC04:56
*** fragatina has joined #openstack-neutron04:56
*** fragatina has quit IRC04:58
*** fragatina has joined #openstack-neutron04:58
*** wolverineav has quit IRC05:02
*** wolverineav has joined #openstack-neutron05:03
*** fragatina has quit IRC05:03
*** fragatina has joined #openstack-neutron05:04
*** fragatina has quit IRC05:11
*** fragatina has joined #openstack-neutron05:11
*** fragatina has quit IRC05:13
*** fragatina has joined #openstack-neutron05:13
*** fragatina has quit IRC05:16
*** fragatina has joined #openstack-neutron05:16
*** fragatina has quit IRC05:17
*** fragatina has joined #openstack-neutron05:17
*** fragatina has quit IRC05:19
*** fragatina has joined #openstack-neutron05:19
*** fragatina has quit IRC05:27
*** fragatina has joined #openstack-neutron05:28
*** dconde has joined #openstack-neutron05:28
*** iyamahat has joined #openstack-neutron05:29
*** fragatina has quit IRC05:30
*** fragatina has joined #openstack-neutron05:30
*** fragatina has quit IRC05:32
*** fragatina has joined #openstack-neutron05:32
*** fragatina has quit IRC05:36
*** fragatina has joined #openstack-neutron05:36
*** yamamoto has joined #openstack-neutron05:42
*** Bhujay has joined #openstack-neutron05:43
*** Bhujay has quit IRC05:44
*** Bhujay has joined #openstack-neutron05:45
*** fragatina has quit IRC05:45
*** Bhujay has quit IRC05:46
*** fragatina has joined #openstack-neutron05:46
*** Bhujay has joined #openstack-neutron05:46
*** markvoelker has joined #openstack-neutron05:48
*** yamamoto has quit IRC05:49
*** fragatina has quit IRC05:51
*** fragatina has joined #openstack-neutron05:52
*** fragatina has quit IRC05:59
*** mdih has joined #openstack-neutron05:59
*** markvoelker has quit IRC05:59
*** fragatina has joined #openstack-neutron05:59
*** abaindur has joined #openstack-neutron05:59
*** abaindur has quit IRC06:00
*** abaindur has joined #openstack-neutron06:00
*** fragatina has quit IRC06:01
*** fragatina has joined #openstack-neutron06:01
*** ratailor has joined #openstack-neutron06:13
*** fragatina has quit IRC06:13
*** fragatina has joined #openstack-neutron06:14
*** fragatina has quit IRC06:18
*** fragatina has joined #openstack-neutron06:18
*** fragatina has quit IRC06:22
*** fragatina has joined #openstack-neutron06:23
*** Dinesh_Bhor has quit IRC06:23
*** fragatina has quit IRC06:23
*** fragatina has joined #openstack-neutron06:24
*** jangutter has joined #openstack-neutron06:24
*** fragatina has quit IRC06:25
*** gcheresh_ has joined #openstack-neutron06:25
*** fragatina has joined #openstack-neutron06:25
*** fragatina has quit IRC06:26
*** fragatina has joined #openstack-neutron06:26
*** fragatina has quit IRC06:27
*** fragatina has joined #openstack-neutron06:27
*** itlinux_ has joined #openstack-neutron06:28
*** wolverineav has quit IRC06:28
*** wolverineav has joined #openstack-neutron06:29
*** jangutter has quit IRC06:29
*** fragatina has quit IRC06:29
*** fragatina has joined #openstack-neutron06:29
*** itlinux has quit IRC06:30
*** fragatina has quit IRC06:31
*** fragatina has joined #openstack-neutron06:32
*** fragatina has quit IRC06:32
*** wolverineav has quit IRC06:33
*** fragatina has joined #openstack-neutron06:33
*** brault has quit IRC06:33
*** _fragatina_ has joined #openstack-neutron06:34
*** yamahata has joined #openstack-neutron06:36
*** _fragatina_ has quit IRC06:37
*** _fragatina_ has joined #openstack-neutron06:37
*** fragatina has quit IRC06:38
*** _fragatina_ has quit IRC06:44
*** fragatina has joined #openstack-neutron06:44
*** fragatina has quit IRC06:47
*** fragatina has joined #openstack-neutron06:48
*** fragatina has quit IRC06:48
*** itlinux_ has quit IRC06:48
*** fragatina has joined #openstack-neutron06:48
*** fragatina has quit IRC06:49
*** fragatina has joined #openstack-neutron06:50
*** fragatina has quit IRC06:52
*** Dinesh_Bhor has joined #openstack-neutron06:53
*** fragatina has joined #openstack-neutron06:53
*** rcernin has quit IRC06:59
*** Bhujay has quit IRC07:01
*** ccamposr has joined #openstack-neutron07:01
*** mosulica has joined #openstack-neutron07:02
*** lajoskatona has quit IRC07:04
*** fragatina has quit IRC07:07
*** fragatina has joined #openstack-neutron07:08
*** yamamoto has joined #openstack-neutron07:08
*** moshele has joined #openstack-neutron07:08
*** Bhujay has joined #openstack-neutron07:10
*** Bhujay has quit IRC07:11
*** Bhujay has joined #openstack-neutron07:11
*** yamamoto has quit IRC07:13
*** dconde has quit IRC07:14
*** fragatina has quit IRC07:18
*** fragatina has joined #openstack-neutron07:19
*** lajoskatona has joined #openstack-neutron07:19
*** iyamahat has quit IRC07:20
*** fragatina has quit IRC07:25
*** fragatina has joined #openstack-neutron07:26
*** fragatina has quit IRC07:31
*** fragatina has joined #openstack-neutron07:31
*** mosulica has quit IRC07:40
*** wolverineav has joined #openstack-neutron07:43
*** fragatina has quit IRC07:44
*** fragatina has joined #openstack-neutron07:44
*** wolverineav has quit IRC07:47
*** fragatina has quit IRC07:49
*** _fragatina_ has joined #openstack-neutron07:49
*** velizarx has joined #openstack-neutron07:57
openstackgerritHamdy Khader proposed openstack/neutron-lib master: Add smartnic port type to VNIC_TYPES  https://review.openstack.org/62466007:58
*** _fragatina_ has quit IRC07:59
*** fragatina has joined #openstack-neutron08:00
*** jangutter has joined #openstack-neutron08:01
*** jangutter has quit IRC08:01
*** jangutter has joined #openstack-neutron08:02
*** rpittau has joined #openstack-neutron08:03
openstackgerritSlawek Kaplonski proposed openstack/neutron master: DNM Check FIP configurations with older pyroute2 version  https://review.openstack.org/63051608:10
*** fragatina has quit IRC08:13
*** fragatina has joined #openstack-neutron08:14
*** tmorin has joined #openstack-neutron08:17
*** panda|off is now known as panda08:19
*** fragatina has quit IRC08:22
*** fragatina has joined #openstack-neutron08:23
*** fragatina has quit IRC08:26
*** fragatina has joined #openstack-neutron08:27
*** jpena|off is now known as jpena08:33
*** ralonsoh has joined #openstack-neutron08:36
*** yamamoto has joined #openstack-neutron08:39
*** iyamahat has joined #openstack-neutron08:42
*** yamamoto has quit IRC08:44
*** fragatina has quit IRC08:45
*** fragatina has joined #openstack-neutron08:46
*** fragatina has quit IRC08:46
*** fragatina has joined #openstack-neutron08:46
*** jlibosva has joined #openstack-neutron08:58
*** fragatina has quit IRC09:01
*** abaindur has quit IRC09:01
*** fragatina has joined #openstack-neutron09:01
*** fragatina has quit IRC09:02
openstackgerritLajos Katona proposed openstack/neutron master: Catch OVSFWTagNotFound in firewall.py  https://review.openstack.org/63091009:02
*** fragatina has joined #openstack-neutron09:03
*** fragatina has quit IRC09:04
*** fragatina has joined #openstack-neutron09:05
*** jangutter has quit IRC09:18
*** jangutter has joined #openstack-neutron09:23
*** pcaruana has joined #openstack-neutron09:30
*** wolverineav has joined #openstack-neutron09:36
*** wolverineav has quit IRC09:41
*** ratailor_ has joined #openstack-neutron09:52
*** ratailor has quit IRC09:55
openstackgerritAdit Sarfaty proposed openstack/neutron-fwaas master: FWaaS v1->v2 DB migration  https://review.openstack.org/61947909:56
*** Dinesh_Bhor has quit IRC09:56
*** davidsha has joined #openstack-neutron10:00
*** robbbe has joined #openstack-neutron10:03
*** Dinesh_Bhor has joined #openstack-neutron10:05
*** Dinesh_Bhor has quit IRC10:06
*** yamamoto has joined #openstack-neutron10:09
*** yamamoto has quit IRC10:12
*** yamamoto has joined #openstack-neutron10:12
*** fragatina has quit IRC10:23
*** fragatina has joined #openstack-neutron10:24
*** fragatina has quit IRC10:28
*** fragatina has joined #openstack-neutron10:29
*** mdih has quit IRC10:31
*** salmankhan has joined #openstack-neutron10:33
*** lpetrut has joined #openstack-neutron10:39
*** moshele has quit IRC10:41
*** salmankhan has quit IRC10:42
*** salmankhan has joined #openstack-neutron10:42
*** brtknr has joined #openstack-neutron10:42
*** salmankhan has quit IRC10:46
*** macza has joined #openstack-neutron10:50
openstackgerritSlawek Kaplonski proposed openstack/neutron master: Support iproute2 4.15 in l3_tc_lib  https://review.openstack.org/63077310:51
*** salmankhan has joined #openstack-neutron10:53
*** brault has joined #openstack-neutron10:54
*** macza has quit IRC10:54
*** jangutter has quit IRC11:03
*** jangutter has joined #openstack-neutron11:04
*** ratailor__ has joined #openstack-neutron11:10
*** ratailor_ has quit IRC11:12
*** sapd1_x has quit IRC11:16
*** openstackgerrit has quit IRC11:22
*** moshele has joined #openstack-neutron11:24
*** jangutter has quit IRC11:26
*** jangutter has joined #openstack-neutron11:27
*** openstackgerrit has joined #openstack-neutron11:34
openstackgerritLucas Alvares Gomes proposed openstack/networking-ovn master: Functional: Workaround database failures  https://review.openstack.org/62924811:34
openstackgerritYang JianFeng proposed openstack/neutron master: Improve port dhcp Provisioning  https://review.openstack.org/62683011:46
*** tbachman has quit IRC11:51
*** fragatina has quit IRC11:54
*** jangutter has quit IRC11:54
*** jangutter has joined #openstack-neutron11:55
*** rpittau is now known as rpittau|lunch11:59
openstackgerritRodolfo Alonso Hernandez proposed openstack/neutron master: Remove IPv6 addresses in dnsmasq leases file  https://review.openstack.org/63095112:01
*** rpittau has joined #openstack-neutron12:02
*** jpena is now known as jpena|lunch12:03
*** rpittau is now known as rpittau|lunch12:05
*** rpittau|lunch has quit IRC12:14
*** rpittau|lunch has joined #openstack-neutron12:15
*** jangutter has quit IRC12:17
*** jangutter has joined #openstack-neutron12:18
*** rpittau|lunch has quit IRC12:29
*** rpittau|lunch has joined #openstack-neutron12:29
*** jangutter has quit IRC12:32
*** rpittau|lunch has quit IRC12:39
*** ratailor__ has quit IRC12:40
*** Luzi has joined #openstack-neutron12:41
*** panda is now known as panda|lunch12:47
*** crazzy has joined #openstack-neutron12:48
*** mosulica has joined #openstack-neutron12:58
*** jpena|lunch is now known as jpena12:59
*** rpittau has joined #openstack-neutron13:03
*** jangutter has joined #openstack-neutron13:08
*** wolverineav has joined #openstack-neutron13:12
*** wolverineav has quit IRC13:16
*** tbachman has joined #openstack-neutron13:19
*** tbachman has quit IRC13:23
*** dave-mccowan has joined #openstack-neutron13:28
*** tbachman has joined #openstack-neutron13:30
*** boden has joined #openstack-neutron13:30
*** boden has quit IRC13:39
*** boden has joined #openstack-neutron13:39
openstackgerritMerged openstack/neutron-dynamic-routing master: Remove setting of DEVSTACK_GATE_EXERCISES  https://review.openstack.org/58315613:42
openstackgerritMerged openstack/neutron-dynamic-routing master: Improve patching of sys.argv  https://review.openstack.org/57573213:42
*** panda|lunch is now known as panda13:43
openstackgerritJan Gutter proposed openstack/os-vif master: Clean up versioned object backlevelling code  https://review.openstack.org/63097613:48
*** velizarx has quit IRC13:50
*** jangutter_ has joined #openstack-neutron13:52
*** jangutter has quit IRC13:55
*** aojea has joined #openstack-neutron13:56
*** njohnston has joined #openstack-neutron13:56
*** tidwellr has joined #openstack-neutron13:56
*** jangutter_ is now known as jangutter13:58
*** mvkr has quit IRC13:58
*** mlavalle has joined #openstack-neutron13:58
*** dconde has joined #openstack-neutron13:59
*** velizarx has joined #openstack-neutron13:59
*** goldyfruit has joined #openstack-neutron14:01
*** mriedem has joined #openstack-neutron14:10
*** mmethot has joined #openstack-neutron14:12
slaweqmriedem: hi14:18
slaweqmriedem: can You take a look at https://bugs.launchpad.net/neutron/+bug/1809878 and tell me if that makes sense for You?14:19
openstackLaunchpad bug 1809878 in neutron "[RFE] Move sanity-checks to neutron-status CLI tool" [Wishlist,New] - Assigned to Slawek Kaplonski (slaweq)14:19
mriedemslaweq: seems fine to me14:20
slaweqmriedem: thx14:20
slaweqthat should be good for our drivers team to approve this RFE then :)14:20
*** otherwiseguy has joined #openstack-neutron14:30
openstackgerritLucas Alvares Gomes proposed openstack/networking-ovn master: Functional: Workaround database failures  https://review.openstack.org/62924814:30
openstackgerritRodolfo Alonso Hernandez proposed openstack/neutron master: Implement add_tc_qdisc and list_tc_qdiscs using pyroute2  https://review.openstack.org/62285914:30
openstackgerritBence Romsics proposed openstack/neutron master: Class to represent Placement state and sync  https://review.openstack.org/58831914:44
openstackgerritBence Romsics proposed openstack/neutron master: WIP Placement reporting service plugin  https://review.openstack.org/58067214:44
openstackgerritBence Romsics proposed openstack/neutron master: Drive binding by placement allocation  https://review.openstack.org/57478314:44
openstackgerritBence Romsics proposed openstack/neutron master: New agent attribute: resources_synced  https://review.openstack.org/63099914:44
openstackgerritBence Romsics proposed openstack/neutron-lib master: New agent attribute: resources_synced  https://review.openstack.org/62621014:45
*** rossella_s has quit IRC14:46
*** mchlumsky has joined #openstack-neutron14:50
*** moshele has quit IRC14:53
lajoskatonamlavalle: Next week is my for neutron bug deputy, could you please add me to the neutron-bugs group in launchpad?14:54
mlavallelajoskatona: yes, I'll do it know14:54
lajoskatonamlavalle: thanks, as see that is the only missing right for bug deputy, am I right?14:55
mlavallelajoskatona: yes14:55
*** tidwellr has quit IRC14:57
*** tbachman has quit IRC14:58
lajoskatonamlavalle: ok14:58
*** rossella_s has joined #openstack-neutron14:59
*** mvkr has joined #openstack-neutron15:00
*** tbachman has joined #openstack-neutron15:01
*** Bhujay has quit IRC15:02
*** mchlumsky has quit IRC15:11
*** tbachman has quit IRC15:12
*** mchlumsky has joined #openstack-neutron15:12
*** dconde has quit IRC15:14
*** dconde1 has joined #openstack-neutron15:14
*** dconde1 has quit IRC15:15
*** dconde has joined #openstack-neutron15:15
*** jangutter has quit IRC15:18
*** tbachman has joined #openstack-neutron15:18
*** jangutter_ has joined #openstack-neutron15:19
*** ihar has joined #openstack-neutron15:24
*** hongbin has joined #openstack-neutron15:28
*** jackivanov has joined #openstack-neutron15:31
jackivanovHi everybody! I deployed neutron dvr and openvswitch. The ports that are attached to br-int (qr, tap and others) have tags. Where those tags are coming from? Is there a pool configured?15:33
jlibosvajackivanov: they are generated dynamically per network per node. so it may happen ports on the same neutron network can have different internal tags on different nodes15:34
openstackgerritJan Gutter proposed openstack/os-vif master: Clean up versioned object backlevelling code  https://review.openstack.org/63097615:36
*** tbachman has quit IRC15:36
jackivanovjlibosva: Aha, I see, thanks. So, is there a chance of a loop if the tag is the same that on the provider interface? The provider network goes tagged, so vlan termination happens on the servers. Some time ago, I've got a loop, but after I deleted the interface that attached to br-int (qr-something, which has the same vlan tag as my provider), the loop has gone.15:39
mriedemis it possible to get an e-r query for bug 1811515?15:39
openstackbug 1811515 in neutron "SSH to FIP fails in CI jobs" [Critical,Confirmed] https://launchpad.net/bugs/181151515:39
*** aojea has quit IRC15:40
mriedemis it this? http://logs.openstack.org/43/628943/2/check/tempest-full/07fe3fe/controller/logs/screen-q-l3.txt.gz?level=ERROR#_Jan_13_17_57_18_30287715:40
*** tbachman has joined #openstack-neutron15:41
jackivanovjlibosva: moreover, there are 2 neutron network nodes, and both had 2 qr-interfaces in br-in with the same tag id. I deleted only on the first node15:43
openstackgerritSlawek Kaplonski proposed openstack/neutron master: Support iproute2 4.15 in l3_tc_lib  https://review.openstack.org/63077315:43
*** jangutter_ is now known as jangutter15:44
*** Luzi has quit IRC15:44
mriedemi guess that shows up in a lot of non-failed jobs as well...15:45
*** salmankhan has quit IRC15:45
*** fragatina has joined #openstack-neutron15:46
slaweqmriedem: in most cases it is error like in http://logs.openstack.org/43/628943/2/check/tempest-full/07fe3fe/controller/logs/screen-q-l3.txt.gz?level=ERROR#_Jan_13_17_57_16_87664815:48
*** aojea has joined #openstack-neutron15:48
*** velizarx has quit IRC15:48
slaweqbut yes, that is related to this bug which You mentioned15:49
jlibosvajackivanov: the provider vlan should end on provider bridge configured by bridge_mappings and then translated to an internal vlan between your provider bridge and br-int15:49
jackivanovjlibosva: Yes, it ends on eth0.180, and added to bridge br-ex. br-ex is specified in bridge_mappings. everything works, but I've got that loop for some reason and thus broadcast storm. No idea why it happens, but all that I found is 2 qr-* ports with the same tag, on 2 neutron network nodes, attached to br-int, after deletion of which the loop gone15:52
jackivanovany ideas?15:52
jlibosvajackivanov: the best would be to find to which network those qr- ports belong. any chance they are from the same subnet?15:53
jackivanovno, it's a private project network15:53
*** tbachman has quit IRC15:53
jackivanovand that port is network:router_interface_distributed15:54
jackivanovand different subnets of course15:57
*** hogepodge has joined #openstack-neutron15:58
*** gcheresh_ has quit IRC15:59
hogepodgemlavalle: in case you missed it, there's a post-mortem on the recent failure of the Target K8s cloud that started with an OpenStack Network upgrade failure. I've reached out to Dan to see if he has any feedback for the team about what went wrong. https://medium.com/@daniel.p.woods/on-infrastructure-at-scale-a-cascading-failure-of-distributed-systems-7cff2a3cd2df16:00
*** tbachman has joined #openstack-neutron16:01
dougwighogepodge: indeed, it was an interesting read.16:02
mlavallehogepodge: will take a look. Thanks16:03
*** salmankhan has joined #openstack-neutron16:04
*** mosulica has quit IRC16:05
*** dtantsur has joined #openstack-neutron16:09
dtantsurhi folks! just to double-check our understanding: is it possible for a *bound* and ACTIVE port to not have any fixed_ips?16:10
dtantsurbecause we seem to have such situation: http://paste.openstack.org/show/742657/16:13
dtantsur(this is ironic + routed networks)16:13
hjensas^^ Is with routed provider networks (segments) . I'd expect no IP (IP_ALLOCATION_DEFERRED) until we provide binding_host_id, but that port have a host_id and the agent + bridge_mappings in the agent looks good.16:14
*** jackivanov has quit IRC16:15
*** aojea has quit IRC16:15
hjensasmlavalle: ^^ any ideas?16:15
dtantsurnothing suspicious in dhcp-agent logs, except that it complains about inability to set DHCP options without fixed IPs16:17
*** tbachman has quit IRC16:18
*** munimeha1 has joined #openstack-neutron16:20
dtantsurserver logs look okay to me, nothing about IP addresses though. the port binding seems to come through.16:20
openstackgerritJan Gutter proposed openstack/os-vif master: Clean up versioned object backlevelling code  https://review.openstack.org/63097616:21
*** aojea has joined #openstack-neutron16:23
*** fragatina has quit IRC16:25
*** aojea has quit IRC16:26
*** fragatina has joined #openstack-neutron16:26
hjensasdtantsur: there seem to be very little logging going on in the code.16:29
hjensasdtantsur: maby we can add some to debug?16:30
dtantsurhjensas: we're talking about containers. I can try modifying code, but you never know if it works :D16:30
dtantsurwhere exactly would you like to add something?16:30
hjensasdtantsur: yeah, I know ... podman exec into the container and no accidental syntax errors I guess.16:31
hjensasdtantsur: https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L659-L66416:31
hjensashttps://github.com/openstack/neutron/blob/master/neutron/objects/subnet.py#L406-L40916:31
dtantsurhjensas: both in the server container?16:32
hjensasdtantsur: yes.16:32
dtantsurhjensas: TIL about 'podman mount' btw16:33
dtantsurmuch better than podman exec, since you can work with your local tools with the whole mounted tree16:33
dtantsurincl. applying patches and stuff16:34
hjensasdtantsur: oh, cool. And you can mount even if crashes and repeatedly restarts?16:34
dtantsurhjensas: dunno. I'd prefer not to learn it today :D16:34
dtantsurbut I think it works for powered off containers16:34
*** dklyle has joined #openstack-neutron16:35
*** moshele has joined #openstack-neutron16:40
dtantsuranother attempt16:42
openstackgerritLucas Alvares Gomes proposed openstack/networking-ovn master: Functional: Workaround database failures  https://review.openstack.org/62924816:42
*** tbachman has joined #openstack-neutron16:44
*** tbachman has quit IRC16:44
dtantsurhjensas: deferred_allocation is False: http://paste.openstack.org/show/742666/16:46
*** jangutter has quit IRC16:47
*** iyamahat has quit IRC16:47
*** fragatina has quit IRC16:47
*** fragatina has joined #openstack-neutron16:48
*** fragatina has quit IRC16:48
hjensasdtantsur: ok, so if you create a port, then update it with binding_host_id - the deffered allocation is false. Is also not getting here: https://github.com/openstack/neutron/blob/master/neutron/objects/subnet.py#L409 ?16:48
*** yamahata has quit IRC16:49
dtantsuroh wait, it's actually not the first logging16:49
*** wolverineav has joined #openstack-neutron16:49
hjensasdtantsur: yes, it has the host_id in that log.16:49
*** fragatina has joined #openstack-neutron16:49
*** rpittau has quit IRC16:49
*** salmankhan has quit IRC16:50
*** salmankhan has joined #openstack-neutron16:50
dtantsurhjensas: okay, these are all hits: http://paste.openstack.org/show/742668/16:50
dtantsurdeferred allocation seems empty string?16:51
hjensasdtantsur: it has 'ip_allocation': u'deferred' .16:51
dtantsuryeah, but "deferred_ip_allocation" never becomes True16:52
hjensasdtantsur: yeah, it's either false or ''16:52
*** aojea has joined #openstack-neutron16:52
hjensasdtantsur: then this https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L679-L687 never happens.16:53
*** fragatina has quit IRC16:53
dtantsurright, now the question is why...16:53
*** wolverineav has quit IRC16:53
*** fragatina has joined #openstack-neutron16:53
*** tbachman has joined #openstack-neutron16:54
*** yamamoto has quit IRC16:55
*** mvkr has quit IRC16:56
*** fragatina has quit IRC16:58
*** fragatina has joined #openstack-neutron16:59
dtantsurhjensas: for some reason fixed_ips is set: http://paste.openstack.org/show/742670/16:59
dtantsurI guess this is why it gets skipped16:59
dtantsurokay, so why do we pass it here? ironic does not pass fixed_ips when binding anything..17:00
*** ccamposr has quit IRC17:00
dtantsura question to neutron cores: is it okay to remove is_attr_set from https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L657 ?17:01
dtantsurafter all, empty list is the same thing?17:01
openstackgerritSlawek Kaplonski proposed openstack/neutron master: DNM Check FIP configurations with older privsep version  https://review.openstack.org/63051617:03
slaweqmlavalle: hongbin: https://review.openstack.org/#/c/630516/ - it is the patch which should be run with oslo.privsep 1.30.117:04
slaweqlets see how it will be17:04
dtantsurotherwise we need to figure out what sends fixed_ips=[]...17:04
*** moshele has quit IRC17:04
slaweqmlavalle: hongbin: now I have to leave but I will be at evening to check that17:04
slaweqmlavalle: I will ping You later about that issue17:04
mlavalleslaweq: if you follow the logstash query that mriedem posted in the bug, the problem seems to start inddded on the 9th17:05
*** fragatina has quit IRC17:06
*** iyamahat has joined #openstack-neutron17:06
*** tmorin has quit IRC17:07
*** macza has joined #openstack-neutron17:08
*** davidsha has quit IRC17:10
*** davidsha has joined #openstack-neutron17:11
*** aojea has quit IRC17:11
hjensasdtantsur: looks like _make_port_dict is adding  fixed_ips=[] .17:12
dtantsur\o/17:12
hjensasdtantsur: https://github.com/openstack/neutron/blob/83eb3e16138cbf3cd2df723020c07ac2f65c88ce/neutron/db/db_base_plugin_common.py#L20517:12
hjensasdtantsur: https://github.com/openstack/neutron/blob/630ff78409560296e66a127a6356df44b64d8548/neutron/plugins/ml2/plugin.py#L1428-L143517:12
*** macza_ has joined #openstack-neutron17:13
dtantsurlovely! so I guess I was right. lemme try a patch17:13
dtantsuris neutron using storyboard or launchpad?17:13
*** macza has quit IRC17:14
hjensasdtantsur: the _make_port_dict is used all over the place ...17:14
hjensasdtantsur: launcpad I belive.17:14
*** wwriverrat has joined #openstack-neutron17:14
*** davidsha has quit IRC17:15
dtantsurhjensas: okay, I hacked this line and it went past that check. but no IPs ended up allocated..17:18
dtantsurI guess I'll continue tomorrow :(17:21
hjensasdtantsur: ok, I'll try to get a reproducer of my own, and will keep digging this evening.17:23
*** wwriverrat has quit IRC17:23
dtantsurhjensas: thanks! you can use metalsmith if you need a simple way to run deployments without nova17:23
*** yamahata has joined #openstack-neutron17:24
hjensasdtantsur: ack, I was hoping the metalsmith work would allow us to not manually match nodes to subnets. And for that this needs to work. :)17:25
dtantsurright :)17:25
*** lajoskatona has quit IRC17:28
*** dtantsur is now known as dtantsur|afk17:28
openstackgerritNate Johnston proposed openstack/neutron master: Utilize bulk port creation ops in ml2 plugin  https://review.openstack.org/62481517:34
*** dtantsur|afk has quit IRC17:36
*** mattw4 has joined #openstack-neutron17:36
*** robbbe has quit IRC17:37
*** jlibosva has quit IRC17:41
*** gcheresh_ has joined #openstack-neutron17:44
*** yamamoto has joined #openstack-neutron17:45
*** yamamoto has quit IRC17:48
*** yamamoto has joined #openstack-neutron17:48
*** yamamoto has quit IRC17:48
openstackgerritBen Nemec proposed openstack/neutron-fwaas master: Inherit parent env values in dsvm-functional  https://review.openstack.org/63103517:50
*** yamamoto has joined #openstack-neutron17:53
*** fragatina has joined #openstack-neutron17:54
*** wolverineav has joined #openstack-neutron17:54
*** wolverineav has quit IRC17:57
*** wolverineav has joined #openstack-neutron17:57
*** jpena is now known as jpena|off17:59
*** pcaruana has quit IRC18:01
*** yamamoto has quit IRC18:01
*** salmankhan1 has joined #openstack-neutron18:04
*** igordc has joined #openstack-neutron18:05
haleybhongbin: ping, can you take a look at https://review.openstack.org/#/c/628691/ ?  would be good to get your approval, thanks18:05
*** gcheresh_ has quit IRC18:06
*** salmankhan has quit IRC18:07
*** yamamoto has joined #openstack-neutron18:07
*** salmankhan1 has quit IRC18:08
*** wwriverrat has joined #openstack-neutron18:12
*** brault has quit IRC18:14
*** brault has joined #openstack-neutron18:15
*** itlinux has joined #openstack-neutron18:16
*** panda is now known as panda|off18:19
*** yamamoto has quit IRC18:19
*** brault has quit IRC18:20
*** yamamoto has joined #openstack-neutron18:23
*** yamamoto has quit IRC18:32
*** yamamoto has joined #openstack-neutron18:33
*** wolverineav has quit IRC18:34
*** wolverineav has joined #openstack-neutron18:35
haleybmlavalle: ping.  i know i ask a lot of dns questions, but... when i have neutron.conf setup and the port extension loaded, should a booted instance have the fqdn as it's hostname?  i can't seem to make it happen even after tweaking the port and nova objects.  guess i could watch the dhcp reply or metadata reply thinking about it some more...18:35
openstackgerritAndrew Karpow proposed openstack/neutron master: postgresl compatibility for get_l3_agent routines  https://review.openstack.org/63104618:35
*** yamamoto has quit IRC18:36
*** yamamoto has joined #openstack-neutron18:37
*** ralonsoh has quit IRC18:38
hongbinhaleyb: ack18:39
*** wolverineav has quit IRC18:40
*** wolverineav has joined #openstack-neutron18:43
*** yamamoto has quit IRC18:45
*** yamamoto has joined #openstack-neutron18:45
*** wolverineav has quit IRC18:48
*** goldyfruit has quit IRC18:58
*** yamamoto has quit IRC18:59
*** wolverineav has joined #openstack-neutron19:02
*** yamamoto has joined #openstack-neutron19:05
*** yamamoto has quit IRC19:09
*** yamamoto has joined #openstack-neutron19:16
*** gcheresh_ has joined #openstack-neutron19:18
*** lpetrut has quit IRC19:38
*** yamamoto has quit IRC19:42
*** yamamoto has joined #openstack-neutron19:42
*** wolverineav has quit IRC19:48
*** wolverineav has joined #openstack-neutron19:48
*** wolverineav has quit IRC19:51
*** wolverineav has joined #openstack-neutron19:51
*** wolverineav has quit IRC19:52
*** wolverineav has joined #openstack-neutron19:52
*** gcheresh_ has quit IRC19:55
*** aojea has joined #openstack-neutron19:56
*** wolverineav has quit IRC19:58
*** brault has joined #openstack-neutron20:06
*** brault has quit IRC20:11
*** aojea has quit IRC20:27
*** yamamoto has quit IRC20:28
*** mvkr has joined #openstack-neutron20:32
*** itlinux_ has joined #openstack-neutron20:36
*** itlinux has quit IRC20:37
*** wolverineav has joined #openstack-neutron20:39
*** itlinux_ has quit IRC20:39
*** tidwellr has joined #openstack-neutron20:43
*** wolverineav has quit IRC20:43
*** yamamoto has joined #openstack-neutron20:49
*** abaindur has joined #openstack-neutron20:51
*** abaindur has quit IRC20:52
*** abaindur has joined #openstack-neutron20:52
*** wolverineav has joined #openstack-neutron20:55
*** aojea has joined #openstack-neutron20:57
*** wolverineav has quit IRC21:00
*** wolverineav has joined #openstack-neutron21:00
*** robbbe has joined #openstack-neutron21:02
*** yamamoto has quit IRC21:02
slaweqmlavalle: ping21:07
mlavalleslaweq: hey21:07
slaweqmlavalle: I updated our hot bug report now21:07
slaweqI was able to reproduce same issue few times with simple script which I put there21:07
slaweqI will continue debugging tomorrow as today I'm really tired already21:08
mlavalleslaweq: that's good21:08
slaweqyes21:08
mlavalleI also left some comments in https://review.openstack.org/#/c/63051621:09
mlavalleabout an hour ago21:09
slaweqyes, I saw it :)21:09
slaweqthx for recheck21:09
slaweqit looks that oslo.privsep with concurrency support exposed this race bug to us :/21:10
slaweqlets check couple of times but it is probably that all will work fine with older privsep lib21:10
mlavalleyes, I was about to propose the same thing21:11
slaweq:)21:11
slaweqok, see You tomorrow21:11
slaweqand have a good night21:11
mlavalleslaweq: I'll keep rechecking the patch and reviewing the log files21:11
mlavalleare those jobs the ones to check?21:11
mlavalleI mean their log files21:12
slaweqbasically yes21:12
slaweqbut I think it might happen in every job in l3 agent logs21:12
mlavalleok, I'll keep checking those21:13
slaweqthx21:13
slaweqgood night21:13
mlavallehave a great night21:13
mlavalleI'll leave comments in the patch21:13
*** yamamoto has joined #openstack-neutron21:17
*** yamamoto has quit IRC21:20
*** salmankhan has joined #openstack-neutron21:23
*** aojea has quit IRC21:29
*** tidwellr has quit IRC21:37
openstackgerritSlawek Kaplonski proposed openstack/neutron master: DNM Try custom pyroute2 lib  https://review.openstack.org/63107521:51
openstackgerritDoug Wiegley proposed openstack/neutron master: Fix performance regression adding rules to security groups  https://review.openstack.org/62869121:51
*** yamamoto has joined #openstack-neutron21:52
*** yamamoto has quit IRC21:52
*** wolverineav has quit IRC21:58
*** wolverineav has joined #openstack-neutron21:58
*** yamamoto has joined #openstack-neutron22:05
*** imacdonn has quit IRC22:06
*** imacdonn has joined #openstack-neutron22:07
*** yamamoto has quit IRC22:14
*** boden has quit IRC22:19
*** yamamoto has joined #openstack-neutron22:23
*** lpetrut has joined #openstack-neutron22:34
*** lpetrut has quit IRC22:38
*** yamamoto has quit IRC22:40
*** yamamoto has joined #openstack-neutron22:41
*** yamamoto has quit IRC22:44
*** brault has joined #openstack-neutron22:45
*** brault has quit IRC22:45
*** rcernin has joined #openstack-neutron22:46
*** tbachman has quit IRC22:58
*** tbachman has joined #openstack-neutron22:58
*** brault has joined #openstack-neutron23:00
*** hongbin has quit IRC23:01
*** brault has quit IRC23:05
*** yamamoto has joined #openstack-neutron23:08
*** yamamoto has quit IRC23:08
*** munimeha1 has quit IRC23:14
*** ijw has joined #openstack-neutron23:21
*** robbbe has quit IRC23:25
*** rossella_s has quit IRC23:45
*** rossella_s has joined #openstack-neutron23:45
*** yamamoto has joined #openstack-neutron23:51
*** salmankhan has quit IRC23:56
*** wolverineav has quit IRC23:56
*** yamamoto has quit IRC23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!