You should do both: disabling the "minikub nginx ingress" addon, AND cleanup the leftover if it does not it for you during disabling (aka. removing the nginx pod AND the Service "default-http-whatever").
minikube addons disable ingress kubectl delete nginx-ingress-controller-7b465d9cf8-ng5lb -n kube-system
$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default hugo-site-5db7568498-8vjtt 1/1 Running 0 98m
default hugo-site-5db7568498-lfw9m 1/1 Running 0 98m
kube-system coredns-5c98db65d4-svgbq 1/1 Running 0 114m
kube-system coredns-5c98db65d4-txr59 1/1 Running 0 114m
kube-system etcd-minikube 1/1 Running 0 113m
kube-system kube-addon-manager-minikube 1/1 Running 0 113m
kube-system kube-apiserver-minikube 1/1 Running 0 112m
kube-system kube-controller-manager-minikube 1/1 Running 0 113m
kube-system kube-proxy-sp6rs 1/1 Running 0 114m
kube-system kube-scheduler-minikube 1/1 Running 0 113m
kube-system kubernetes-dashboard-7b8ddcb5d6-44rnr 1/1 Running 0 114m
kube-system storage-provisioner 1/1 Running 0 114m
kube-system traefik-ingress-controller-668df9b887-nz4fk 1/1 Running 0 108m
$curl -v http://hugo-site.minikube: 30380
... 
$ curl -v http://hugo-site.minikube:30380
* Rebuilt URL to: http://hugo-site.minikube:30380/
* Trying 192.168.99.120...
* TCP_NODELAY set
* Connected to hugo-site.minikube (192.168.99.120) port 30380 (#0)
> GET / HTTP/1.1
> Host: hugo-site.minikube:30380
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Content-Length: 4028
< Content-Type: text/html
< Date: Mon, 29 Jul 2019 13:36:58 GMT
< Etag: "5d385027-fbc"
< Last-Modified: Wed, 24 Jul 2019 12:33:43 GMT
< Server: nginx/1.16.0
<
<!DOCTYPE html>
<html lang="en-us">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<title>My New Hugo Site </title>
Yay! now to get it on port 80 would be awesome
(http://traefik-ui.minikube moved to http://traefik-ui.minikube:30380/dashboard/ as well)
Maybe I should just start over without the ingress addon? Although, you probably know how to fix this. Does the nginx layer in the hugo-site container matter? No, right?
Minikube stop/start doesn't do it.
I could port forward manually on the VirtualBox minikube VM perhaps, but I'd prefer to log in to the traefik container and look at the (NGINX?) settings.
There was one other thing I noticed somehwere: the yaml files obtained some additions, possibly from kubectl:
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"cheddar","namespace":"default"},"spec":{"ports":[{"name":"http","port":80,"targetPor$
creationTimestamp: null
name: cheddar
selfLink: /api/v1/namespaces/default/services/cheddar
This is from a cheddar.yaml (but a little old by now, I don't remember when the annotation was added).
If you want to be able to use the port external port 80, then you mustdo the followings:
- Uninstall Traefik's Helm Chart as well
- Re-Install Traefik with Helm chart, and do not specify a "nodeport" service (let the service to the default value
LoadBalancer) - Use the command
minikube tunnelcommand described here: https://github.com/kubernetes/minikube/blob/master/docs/tunnel.md . This command is able to do the same kind of thing as the "ingress addon": aka. mapping the "service of type loadbalancer" to your host machine, outside minikube. Once the commandminikube tunnelhad been created, then when runningkubcetl get svc --all-namespace, you should see anEXTERNAL-IPset for the service of type "LoadBalancer" pointing to Traefik. This External IP should be reachable from your host machine, and you can points the/etc/hostsentries to this IP (instead of minikube's IP).
Yes, totally: less moving pieces, less issues understanding, less pain maintaining.
Nginx in the "hugo-site" is used as a webserver (aka. serving static files with HTTP protocol). It's working well and is not responsible for any issue here.
For sure, see my previous message about service of type "loadbalancer".
You're almost there: Kubernetes is powerful, but requires a lot of learnings. Also, keep in mind that minikube is only a development tool: it's a not really a valid Kubernetes distribution (it does not pass the conformance test), and I find it (personal opinion only!) very complicated to get started.
I prefer using https://github.com/kubernetes-sigs/kind or k3s (see https://rancher.com/docs/k3s/latest/en/running/#running-in-docker-and-docker-compose), that are basically starting kubernetes clusters inside a docker container: no need for VM, and the network model is far easier to understand.
This is a norma Kubernetes behavior. I think you are confused by the amount of moving pieces= you should strongly check Kubernetes official online doc and tutorials at Learn Kubernetes Basics | Kubernetes to get a better understanding of this.
I really think you are fabulous and very attentive. I am not against using the Helm chart, but it seems like I have to learn another thing. If the nginx addon was hijacking port 80, can't Traefik take it back now?
Funny thing is I don't need a load balancer because I have more than 20 websites with hardly any legitimate traffic. Having K8 will hopefully make me more productive and get more traffic sometime.
I will turn on the statistics option and see if there's anything I can do in return.
Right, by using the Helm Chart, you have to learn how to install/uninstall Helm, and how to provide the custom values. But for me it's still less pain than having to masterize all Kubernetes components 
The "nginx hijacking the port 80" is not doable. Because you are dealing with a VM, which network depends on the host physical machine and the OS: doing something reusable is clearly complicated.
In the history of minikube, the command minikube ip came first, as it's relying on the capability of Virtualbox to create a virtual network interface bound between your host and the minikube VM. This interface has the public IP of 192.168.xx.yy you used. Using the port 80 of this machine requires special rights, hence it is a specific feature of minikube which forces you to use Nginx, and that not exist in real life Kubernetes clusters.
Please note that a "LoadBalancer" is not only aimed at balancing traffic through nodes. it's also a layer os network isolation between external and internal networks, which is why the "service" of type "load balancer" in Kubernetes tends to be widely used, as it separate concerns and provides a better lifecycle management (less things to worry about: no need to get a free port, no need for specific rights, etc.).
Are you running this on a single machine? Is there a particular reason for you to use Kubernetes instead a single Docker? (I like things simple 
)
Thanks a lot! If you are able to participate in advocating Traefik to others, and sharing the knowledge you acquired on Kubernetes integration, it would be really nice!
I ike simple things too. To be perfectly honest, I have had numerous projects where NGINX was used a reverse proxy and Let's Encrypt was used. So it would make sense to automate that. I can imagine the Traefik dashboard have a dropdown "add LE certifcates" or "from CA" or BYOC. With the pods moving about and life cycling, it would make sense to have some tool manage this. I don't know about the other uses of Traefik, but when I saw this:
-
Traefik is a fully featured ingress controller (Let’s Encrypt, secrets, http2, websocket), and it also comes with commercial support by Containous.
I had hoped the LoadBalancer would be part of the default ingress and all that bashing would be included. I'm not sure it is. Like I said, I'm just being honest. I hope that may give you an idea of what kind of lazy dumbos may be using your software.
Traefik is totally a replacement for the couple Nginx (reverse-proxy part) + Certbot for LE certificates.
The dashboard UI of Traefik is not meant to interact with its configuration, but only to "visualize" it.
Please note that LE support in Traefik automates everything: you do NOT have to ask for certificates creation neither renewal: everything is done by Traefik itself (which is perfect for a lazy person as I am).
I'm not sure to understand: The Service of type "LoadBalancer" is part of minikube as any other Kubernetes distribution?
I mean part of the traefik ingress, like " * Re-Install Traefik with Helm chart, and do not specify a "nodeport" service (let the service to the default value LoadBalancer )" So it is part of the helm chart but not the tutorial I used?
I hope I'm getting closer to what you describe that it does.
Why can't you log in to the pod actually? (using minikube ssh then docker exec -it bin/(b)ash? I'm now trying to work around the port thing, on the VBox, if it works I will move on so as to understand some things.
I just "scaled" to 4 new hugo websites, that's a good end for today.
Now I'm thinking, why did the cheeses work on stilton.minikube etc. ?
On VirtualBox, I port forward 8080 to the NodePort on minikube. Now I can for example visit wensleydale.minikube:8080 from the VBox host.
I used this same setup with a mere NGINX on a Ubuntu VM: on my router I forward http to 8080 on the server where VirtualBox runs, which for example forwarded it to the port where NGINX was listening. Now I forward it there to the NodePort:
internet -> router:80 -> server:8080 -> VBox:8080 -> VM:30975
I have some domains pointing there, including the one that previously used the NGINX server (I made a new service, deployment and ingress for it too). Unfortunately, the services aren't accessible from the outside world.
I would guess the header including the domain name was lost between server:8080 and VBox:8080 but this works in the other set up. I don't think it is lost here either: VBox:8080 -> VM:30975 because that works on the host browser although that looks up the IP address in /etc/hosts. Any suggestion would be appreciated.
By default minikube creates two network adapters in VirtualBox. One NAT and one Host-only network adapter. The NAT can be used to port forward from the host to the Minikube guest.
However, can it be that Traefik Ingress goes on the Host-only network? Because access via /etc/hosts does work for the Traefik Ingress, but access from the internet doesn't. Or, that Traefik does so given what @dduportal alluded to above:
cf. here too
So Traefik is unable to fully replace the NGINX ingress (addon) that is shipped with minikube, is it? (puts it on the host-only network instead) Although the helm chart is able to do that?
Editing the Helm chart's 1000 parameters is too complex. If you then add TOML, it becomes unfeasible.
I also see that args can be passed as part of a Deployment.
Hi @tmpl, I'm really confused by what you are trying to achieve and the challenges caused by Minikube here.
My understanding is that you want to run a few websites (5-20 ) on a single Ubuntu server, with HTTPS enabled with the help of LE certificates right?
What is the reason for you to use Kubernetes instead of a single Docker Engine?
And why using minikube, which is a development-purposed tool, instead of a real managed distribution of Kubernetes? Escpecially when minikube adds a layer of virtualization (vbox) which might not need to be used).
Well, I'm basically learning. I have read Kubernetes in Action up to Chapter 5 and found it necessary to see if I can make a somewhat usable learning environment. In my work I use many different kinds of web apps, websites and web services. They are also commonly expendable. However, for example, adding the LE certificates is something that is a hassle. Using docker I often find the containers grow because I use a lot of dependencies. When developing many apps at the same time, I prefer to have them separate on a lightweight VM. With Kubernetes I can also control the cluster from outside VirtualBox without installing a lot of software on my Mac host. VM's can be paused and backed upped. I don't need to open ports on the Mac host either, only on VirtualBox. Often when I don't work on a project for some time, I find the configuration has become stale, and Docker can mix dependencies which I would have to keep in check. With K8 everything looks the same from the outside and it can go to other places (like a cloud VPS when necessary). My Mac also has a lot of resurces left over, while I find myself paying a few hundred dollars to web providers every month to do the same: host stale websites. I would like to have 3 2-CPU nodes, but not one for load balancing: there is little traffic on these websites.
I think you could add a lot of what you replied here to the Kubernetes tutorial. I thought about doing that myself, but it would have to end up with working solutions. K8 may be one way where people come to Traefik from. But, given that the TOML and Helm chart go into the ConfigMap, I would use that in the tutorial because the Helm chart has a large number of parameters. In the end what we worked through here, works, except that the ingress addon can apparently go on the VBox NAT network while the Traefik deployment puts it on the host only network (if I'm correct). So the step of changing that, it may be in the arguments that have to go into the ConfigMap and that may already be in the tutorial implicitly.
I am sorry but I really don't understand, there are too much things here. I understand that you are in the process of learning, but there are too much information at the same time to help.
Based on my understanding of your use case, you should totally start with simple things, and then iterate one thing at a time.
I understand the eagerness to learn new stuff, especially Kubernetes. But this tool is really made for big production setup, because it has a cost for the first step, for the upgrade, etc. ("day 0 with Kubernetes is hard, day 1 is cool, day 2 is a nightmare, wait for day 10 before being happy"). And your use case looks a better fit to Docker + good practices (always used image built and pushed from the hub, use Traefik as single reverse proxy with LE enabled, run containers in read-only, update often, etc.).
A setup with a single Docker Engine and docker-compose could work easily, please check https://containous.github.io/slides/online-meetup-201901.
I tried my best helping you, and I might had understand all the concerns you have here. If you feel that Traefik is not a fit in your case, do not hesitate to go back to the default nginx
You could also add a LoadBalancer to the tutorial yaml's. I understand what you say and I'm thankful for your help. I found K8 day 1 a breeze to be honest and an enormous move forward from separate Docker's. K8 with the minikube tutorial is not so difficult. The ease led me to try Traefi right away.
I find trow (private docker repo/hub) and traefik more complex. I will try it with NGINX' ingress though since that's included with Minikube and possibly offers LE scripts (even semi-manual for 5-20 domains could be feasible). If it doesn't work, I will find help at containero.us
1 Like