How to control the behavior of enb?

[JC1O24]

For example, I do not want to start enb when the container started, what can I do?
I try to remove this line:
https://github.com/cncf/cnf-testbed/blob/master/examples/use_case/gogtp-k8s/k8s_bridged_networks/gogtp/templates/deployment.yaml#L19
But no luck, the container of enb pod started then become failed, any ideas?

[michaelspedersen]

Hi @JC1O24. Instead of removing the line, try replacing it with tail -f /dev/null or another "simple" task that will keep the container occupied

[DAIGOR1024]

Hi @JC1O24. Instead of removing the line, try replacing it with tail -f /dev/null or another "simple" task that will keep the container occupied

Tried. It works. Thank you so much!

[JC1O24]

Hi, @michaelspedersen. I manually start enb process by /usr/local/bin/enb -config /root/enb.yml.
Then I kill it by pid running kill -9 {pid_of enb}. After around 3 times of starting and stopping, an error occurs like:

[eNB] 2020/12/01 09:46:42 Established S1-MME connection with 172.21.1.12:36412
[eNB] 2020/12/01 09:46:42 FATAL: failed to add device: gtp-enb: file exists

Then I can not start the enb process again. Any ideas? Thanks!

The process after starting:

PID   USER     TIME  COMMAND
    1 root      0:00 {enb_setup.sh} /bin/sh ./root/enb_setup.sh
    7 root      0:00 tail -f /dev/null
   14 root      0:00 /bin/sh
   33 root      0:00 /usr/local/bin/enb -config /root/enb.yml
   43 root      0:00 ps

The process after stoping:

PID   USER     TIME  COMMAND
    1 root      0:00 {enb_setup.sh} /bin/sh ./root/enb_setup.sh
    7 root      0:00 tail -f /dev/null
   14 root      0:00 /bin/sh
   75 root      0:00 ps -A

[michaelspedersen]

@JC1O24 If you are running in a container I would try redeploying it. It looks like the issue is ENB trying to set up a GTP endpoint and failing to do so. Alternatively you will need to mess around with netlink, which can be a bit difficult directly through the terminal.

[DAIGOR1024]

@JC1O24 If you are running in a container I would try redeploying it. It looks like the issue is ENB trying to set up a GTP endpoint and failing to do so. Alternatively you will need to mess around with netlink, which can be a bit difficult directly through the terminal.

In k8s pod

[michaelspedersen]

Yeah then try redeploying the pod. If that doesn't help, and you don't want to mess around with netlink, you can try rebooting the node

[DAIGOR1024]

Yeah then try redeploying the pod. If that doesn't help, and you don't want to mess around with netlink, you can try rebooting the node

Tried. But this is not what I want. May need further studying

[michaelspedersen]

Yeah then try redeploying the pod. If that doesn't help, and you don't want to mess around with netlink, you can try rebooting the node

Tried. But this is not what I want. May need further studying

Otherwise going forward (while not being entirely sure if possible) it might be worth considering a different signal than SIGKILL, so the app has a chance to do a graceful termination :)

[JC1O24]

Hi @michaelspedersen, when running the pgw, I found this error:

[P-GW] 2020/12/04 03:53:32 Started serving S5-C on 172.25.1.14:2123
[P-GW] 2020/12/04 03:53:32 FATAL: failed to add device: gtp-pgw: operation not supported
[P-GW] 2020/12/04 03:53:32 error reading from Conn 172.25.1.14:2123: read udp 172.25.1.14:2123: use of closed network connection

Also for sgw:

[S-GW] 2020/12/04 03:56:24 Started serving S11 on 172.22.0.13:2123
[S-GW] 2020/12/04 03:56:24 Started serving S5-C on 172.25.1.13:2123
[S-GW] 2020/12/04 03:56:24 FATAL: failed to add device: gtp-sgw-s1: operation not supported
[S-GW] 2020/12/04 03:56:24 error reading from Conn 172.22.0.13:2123: read udp 172.22.0.13:2123: use of closed network connection
[S-GW] 2020/12/04 03:56:24 error reading from Conn 172.25.1.13:2123: read udp 172.25.1.13:2123: use of closed network connection

Also for enb:

[eNB] 2020/12/04 07:03:39 Established S1-MME connection with 172.21.1.12:36412
[eNB] 2020/12/04 07:03:39 FATAL: failed to add device: gtp-enb: operation not supported

Any ideas?

[michaelspedersen]

@JC1O24 Are the bridge networks being created and available inside of the containers? It's not an error I recall ever seeing, and I'm wondering if it is due to the interfaces not being added. Regardless, I'll setup a cluster and try it out myself

[wmnsk]

FYI, I’ve seen it when the kernel GTP-U module is not loaded. So, checking the kernel version(4.12+) and the list of loaded module with lsmod to see if there are gtp and udp_tunnel might help.

[DAIGOR1024]

FYI, I’ve seen it when the kernel GTP-U module is not loaded. So, checking the kernel version(4.12+) and the list of loaded module with lsmod to see if there are gtp and udp_tunnel might help.

Only kernel version 4.12+ can work?

[electrocucaracha]

Hi @michaelspedersen, when running the pgw, I found this error:

[P-GW] 2020/12/04 03:53:32 Started serving S5-C on 172.25.1.14:2123
[P-GW] 2020/12/04 03:53:32 FATAL: failed to add device: gtp-pgw: operation not supported
[P-GW] 2020/12/04 03:53:32 error reading from Conn 172.25.1.14:2123: read udp 172.25.1.14:2123: use of closed network connection

Also for sgw:

[S-GW] 2020/12/04 03:56:24 Started serving S11 on 172.22.0.13:2123
[S-GW] 2020/12/04 03:56:24 Started serving S5-C on 172.25.1.13:2123
[S-GW] 2020/12/04 03:56:24 FATAL: failed to add device: gtp-sgw-s1: operation not supported
[S-GW] 2020/12/04 03:56:24 error reading from Conn 172.22.0.13:2123: read udp 172.22.0.13:2123: use of closed network connection
[S-GW] 2020/12/04 03:56:24 error reading from Conn 172.25.1.13:2123: read udp 172.25.1.13:2123: use of closed network connection

Also for enb:

[eNB] 2020/12/04 07:03:39 Established S1-MME connection with 172.21.1.12:36412
[eNB] 2020/12/04 07:03:39 FATAL: failed to add device: gtp-enb: operation not supported

Any ideas?

@JoeLeeWu In my case, I have a script that waits until the other services are available. Using this approach, I can start the pods in any order.

[wmnsk]

@JoeLeeWu

Only kernel version 4.12+ can work?

Yes, because it depends on Linux Kernel GTP-U introduced around that version.

[JC1O24]

Hi @michaelspedersen, when running the pgw, I found this error:

[P-GW] 2020/12/04 03:53:32 Started serving S5-C on 172.25.1.14:2123
[P-GW] 2020/12/04 03:53:32 FATAL: failed to add device: gtp-pgw: operation not supported
[P-GW] 2020/12/04 03:53:32 error reading from Conn 172.25.1.14:2123: read udp 172.25.1.14:2123: use of closed network connection

Also for sgw:

[S-GW] 2020/12/04 03:56:24 Started serving S11 on 172.22.0.13:2123
[S-GW] 2020/12/04 03:56:24 Started serving S5-C on 172.25.1.13:2123
[S-GW] 2020/12/04 03:56:24 FATAL: failed to add device: gtp-sgw-s1: operation not supported
[S-GW] 2020/12/04 03:56:24 error reading from Conn 172.22.0.13:2123: read udp 172.22.0.13:2123: use of closed network connection
[S-GW] 2020/12/04 03:56:24 error reading from Conn 172.25.1.13:2123: read udp 172.25.1.13:2123: use of closed network connection

Also for enb:

[eNB] 2020/12/04 07:03:39 Established S1-MME connection with 172.21.1.12:36412
[eNB] 2020/12/04 07:03:39 FATAL: failed to add device: gtp-enb: operation not supported

Any ideas?

@JoeLeeWu In my case, I have a script that waits until the other services are available. Using this approach, I can start the pods in any order.

After my testing with Linux version which is 4.12+, the issue is gone.
Thanks. But I am still finding a graceful solution for start and stop enb: #370 (comment)