| *** janki has joined #openstack-zun | 05:00 | |
| *** janki has quit IRC | 12:25 | |
| *** yevi has joined #openstack-zun | 16:24 | |
| *** hongbin has joined #openstack-zun | 16:37 | |
| yevi | what database/table does kuryrlibnetwork use to track pools for the networks it creates when integrated with neutron? I think I have an artifact left over from consectutive rebuilds of zun-compute to the same database and I can't seem to find it | 17:10 |
|---|---|---|
| yevi | nevermind I found it, thank you | 18:16 |
| hongbin | fwiw, kuryr-libnetwork is stateless, it uses neutron as the source of truth, however, docker will persist the network information to etcd | 18:35 |
| *** rm_work_ has joined #openstack-zun | 21:37 | |
| *** rm_work has quit IRC | 21:49 | |
| *** rm_work_ is now known as rm_work | 21:49 | |
| yevi | my problem was that when I was upgrading/building zun in my production environment I had messed something up when I automated it and had to rebuild zun. kury-libnetwork had already created a subnet pool in the neutron database, so after being rebuilt and trying to relaunch containers on that same network it would error and say the subnet pool allready existed. I had no intention of rebuilding the database as it was a production en | 22:50 |
| yevi | vironment so I was just looking for the database entry so I could manually purge it. The neutron database is rather large and I was having troubles finding the right table. | 22:50 |
| yevi | I could have worded my original question better | 22:53 |
| hongbin | yevi: you can try to remove the network in docker: | 22:53 |
| hongbin | yevi: list the docker network: docker network ls | 22:53 |
| hongbin | yevi: find any remove the docker network: docker network rm XXX | 22:53 |
| yevi | it wasn't listed in docker anymore, or at least I didn't see it | 22:53 |
| hongbin | ok | 22:54 |
| hongbin | if it isn't listed in docker, make sure the zun's database entry is removed | 22:54 |
| yevi | docker and kuryr-libnetwork had been rebuilt, but neutron was still tracking it so when kuryr sent the request it errored | 22:54 |
| hongbin | ok, i see | 22:55 |
| yevi | ok, I will look for that next time, thank you | 22:55 |
| hongbin | np | 22:56 |
| yevi | just an odd use case I accidently hit I think | 22:56 |
| hongbin | that is fine | 23:01 |
| hongbin | zun is tracking the list of docker networks in database, just need to make sure it is sync | 23:02 |
| hongbin | if using mysql, enter the database "zun", search the "network" table, find the row and remove it | 23:03 |
| hongbin | let me know if it still doesn't work | 23:03 |
| yevi | looking | 23:08 |
| yevi | ok, I see it there now. but to make it interesting when I rebuilt zun I had deleted the zun database. I had rebuilt etcd, zun-api, zun-wsproxy, zun-compute, kuryr-libnetwork, docker. After that when I had tried to use my provider network again it errored out. I had to fix it by deleting the subnet pool from neutron's db subnetpool table. After that everything starting working | 23:15 |
| yevi | I did not rebuild neutron or mysql, so that entry still existed | 23:15 |
| yevi | It is fixed now | 23:16 |
| yevi | Is there a way to sync what zun is seeing as current networks vs what neutron is seeing in subnetpools table? For example, in the neutron.subnetpools table is all entries are for kuryr subnet pools kuryrpool-x.x.x.x/xx. When I rebuilt the services those entries persisted causing my error | 23:25 |
| yevi | what I mean is when I rebuilt zun and all associated services which meant that the zun db was pretty much empty, nothing told neutron that that subnetwork was deleted and neutron still thought it existed and would not create it again which is expected. I guess I am missing the piece where I should have done some clean up prior to rebuilding zun so kuryr would update neutron appropriately. I don't think this is a problem with how t | 23:55 |
| yevi | hings are done currently, as I can't think of a time when this scenario would normally happen as most deployment methods would account for these. We do not use typical openstack deployment methods, so I think I just ran into a scenario that most will never see and I need to account for in my own deployments. | 23:55 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!