Deploying Traefik on a Linux Service Fabric cluster

javaadpatel07 · July 22, 2019, 7:54pm

Hi,

I'm trying to deploy traefik on a linux service fabric cluster, following this github document:https://github.com/jjcollinge/traefik-on-service-fabric/blob/master/Docs/Deployments/sfctl.MD but I keep getting an error "FABRIC_E_IMAGEBUILDER_VALIDATION_ERROR: The EntryPoint traefik is not found."

I've edited the application manifest and servicemanifest as the document suggested. Also the cluster is secured so I have uncommented the TLS section of the .toml file

Has anyone encountered this before?

solidcloudio · July 23, 2019, 4:45am

Yes, I have, what binary are you using and what does your ServiceManifest Entry point section look like ? Mine looks like this:

<?xml version="1.0" encoding="utf-8"?>
<ServiceManifest Name="TraefikPkg" Version="1.0.0" xmlns="http://schemas.microsoft.com/2011/01/fabric" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <ServiceTypes>
    <!-- This is the name of your ServiceType. 
         The UseImplicitHost attribute indicates this is a guest executable service. -->
    <StatelessServiceType ServiceTypeName="TraefikType" UseImplicitHost="true" />
  </ServiceTypes>
  <!-- Code package is your service executable. -->
  <CodePackage Name="Code" Version="1.0.0">
    <!-- The SetupEntryPoint is an optional element used to specify a
         program to be executed before the service's code is launched. -->
    <EntryPoint>
      <ExeHost>
        <Program>traefik_linux-amd64</Program>
        <Arguments>--configfile=traefik.toml</Arguments>
        <WorkingFolder>CodePackage</WorkingFolder>
        <!-- Uncomment to log console output (both stdout and stderr) to one of the service's working directories. Do not use in production. -->
        <ConsoleRedirection FileRetentionCount="5" FileMaxSizeInKb="2048" />
      </ExeHost>
    </EntryPoint>
  </CodePackage>
  <!-- Config package is the contents of the Config directoy under PackageRoot that contains an 
       independently-updateable and versioned set of custom configuration settings for your service. -->
  <ConfigPackage Name="Config" Version="1.0.0" />
  <Resources>
    <Endpoints>
      <!-- This endpoint is used by the communication listener to obtain the port on which to 
           listen. Please note that if your service is partitioned, this port is shared with 
           replicas of different partitions that are placed in your code. -->
      <Endpoint Name="TraefikTypeEndpoint" UriScheme="http" Port="8029"  Protocol="http" />
      <Endpoint Name="TraefikTypeAPIEndpoint" UriScheme="http" Port="8030" Protocol="https" />
    </Endpoints>
  </Resources>
</ServiceManifest>

traefik_linux-amd64 is in my code package and its the amd64 image from the release you are using..

solidcloudio · July 23, 2019, 5:24am

The other issue I had setting this up was the Cluster connection.. You said you cluster is secured? If you are deploying Traefik to a Node inside service fabric, it will access your services over https://localhost:19080

So, to get this to work, you need to add an addtional self-signed certificate with the DNS name for localhost added to your Keyvault, and then added to the Service Fabric Cluster as a Read-Only certificate then your toml will look like this:

# Service Fabric Management Endpoint
clusterManagementUrl = "https://localhost:19080"

# Service Fabric Management Endpoint API Version
apiVersion = "3.0"
refreshSeconds = 10

# Enable TLS connection.
#
# Optional
#
[serviceFabric.tls]
  cert = "certs/localClusterCert-rte.crt"
  key = "certs/localClusterCert-rte.key"
  insecureskipverify = true

Extract these certs using the steps outlined:

The Traefik SF docs say:

Træfik must authenticate to the Service Fabric management API. Currently, you can only do this using a PEM formatted client certificate. If you only have a .pfx certificate you will need to convert it using the following commands:

Extract the private key from the .pfx file

openssl pkcs12 -in $pfxCertFilePath -nocerts -nodes -out "$clientCertOutputDir\servicefabric.key" -passin pass:$certPass

Extract the certificate from the .pfx file

openssl pkcs12 -in $pfxCertFilePath -clcerts -nokeys -out "clientCertOutputDir\servicefabric.crt" -passin pass:$certPass

Træfik only requires read-only access to the Service Fabric API and thus you should use a Read-Only certificate.

It fails to mention it will be resolving via localhost and thats the certificate your going to need.

You can see the traefik logs on your fabric node under /mnt/sfroot/_App/Traefik_App/Trafeik_code/logs....

That's it, Traefik should be able to explore your SFCluster!

your welcome, it took me about a week to figure this crap out.. still working on the routing rules..

javaadpatel07 · July 23, 2019, 11:44am

I noticed that that your reference for is traefik_linux-amd64, I was using the windows one but I've now changed to the download the linux using:
curl -LOk https://github.com/containous/traefik/releases/download/v1.7.12/traefik_linux-amd64.exe > traefik.exe

My .toml looks like:

################################################################
# Global configuration
################################################################

# Enable debug mode
#
# Optional
# Default: false
#
debug = true

# Traefik logs file
# If not defined, logs to stdout
#
# Optional
# 
# [traefiklog]
# filePath = "log/traefik.log"

# Log level
#
# Optional
# Default: "ERROR"

logLevel = "INFO"

# Entrypoints to be used by frontends that do not specify any entrypoint.
# Each frontend can specify its own entrypoints.
#
# Optional
# Default: ["http"]
#
defaultEntryPoints = ["http", "https"]

# Entrypoints definition
#
# Optional
# Default:
[entryPoints]
[entryPoints.http]
address = ":8081"
[entryPoints.traefik]
address = ":8080"

# Enable access logs
# By default it will write to stdout and produce logs in the textual
# Common Log Format (CLF), extended with additional fields.
#
# Optional
#
# [accessLog]

# Sets the file path for the access log. If not specified, stdout will be used.
# Intermediate directories are created if necessary.
#
# Optional
# Default: os.Stdout
#
# filePath = "/path/to/log/log.txt"

# Format is either "json" or "common".
#
# Optional
# Default: "common"
#
# format = "common"

################################################################
# API definition
################################################################

[api]
  # Name of the related entry point
  #
  # Optional
  # Default: "traefik"
  #
  entryPoint = "traefik"

  # Enabled Dashboard
  #
  # Optional
  # Default: true
  #
  dashboard = true

  # Enable debug mode.
  # This will install HTTP handlers to expose Go expvars under /debug/vars and
  # pprof profiling data under /debug/pprof.
  # Additionally, the log level will be set to DEBUG.
  #
  # Optional
  # Default: false
  #
  debug = true

################################################################
# Service Fabric provider
################################################################

# Enable Service Fabric configuration backend
[servicefabric]

# Service Fabric Management Endpoint
clustermanagementurl = "https://localhost:19080"
# Note: use "https://localhost:19080" if you're using a secure cluster

# Service Fabric Management Endpoint API Version
apiversion = "3.0"

# Enable TLS connection.
#
# Optional
#
[serviceFabric.tls]
  cert = "certs/servicefabric.crt"
  key = "certs/servicefabric.key"
  insecureskipverify = true

and, my servicemanifest:

<?xml version="1.0" encoding="utf-8"?>
<ServiceManifest Name="TraefikPkg" Version="1.0.0" xmlns="http://schemas.microsoft.com/2011/01/fabric" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <ServiceTypes>
    <!-- This is the name of your ServiceType. 
         The UseImplicitHost attribute indicates this is a guest executable service. -->
    <StatelessServiceType ServiceTypeName="TraefikType" UseImplicitHost="true" />
  </ServiceTypes>
  <!-- Code package is your service executable. -->
  <CodePackage Name="Code" Version="1.0.0">
    <!-- The SetupEntryPoint is an optional element used to specify a
         program to be executed before the service's code is launched. -->
    <EntryPoint>
      <ExeHost>
        <Program>traefik_linux-amd64</Program>
        <Arguments>--configfile=traefik.toml</Arguments>
        <WorkingFolder>CodePackage</WorkingFolder>
        <!-- Uncomment to log console output (both stdout and stderr) to one of the
             service's working directories. Do not use in production. -->
        <ConsoleRedirection FileRetentionCount="5" FileMaxSizeInKb="2048" />
      </ExeHost>
    </EntryPoint> 
  </CodePackage>
  <!-- Config package is the contents of the Config directoy under PackageRoot that contains an 
       independently-updateable and versioned set of custom configuration settings for your service. -->
  <ConfigPackage Name="Config" Version="1.0.0" />
  <Resources>
    <Endpoints>
      <!-- This endpoint is used by the communication listener to obtain the port on which to 
           listen. Please note that if your service is partitioned, this port is shared with 
           replicas of different partitions that are placed in your code. -->
      <Endpoint Name="TraefikTypeEndpoint" UriScheme="http" Port="8081" />
      <Endpoint Name="TraefikTypeAPIEndpoint" UriScheme="http" Port="8080" />
    </Endpoints>
  </Resources>
</ServiceManifest>

and finally application manifest:

<?xml version="1.0" encoding="utf-8"?>
<ApplicationManifest ApplicationTypeName="TraefikType" ApplicationTypeVersion="1.0.0" xmlns="http://schemas.microsoft.com/2011/01/fabric" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <Parameters>
    <Parameter Name="Traefik_InstanceCount" DefaultValue="-1" />
  </Parameters>
  <!-- Import the ServiceManifest from the ServicePackage. The ServiceManifestName and ServiceManifestVersion 
       should match the Name and Version attributes of the ServiceManifest element defined in the 
       ServiceManifest.xml file. -->

  <!-- UNCOMMENT TO ENABLE WATCHDOG -->
  <!--
  <ServiceManifestImport>
        <ServiceManifestRef ServiceManifestName="Watchdog" ServiceManifestVersion="1.0.0" />
        <ConfigOverrides />
        <Policies>
          <RunAsPolicy CodePackageRef="Code" UserRef="AdminUser" EntryPointType="All" />
        </Policies>
  </ServiceManifestImport>
  -->

  <ServiceManifestImport>
    <ServiceManifestRef ServiceManifestName="TraefikPkg" ServiceManifestVersion="1.0.0" />
    <ConfigOverrides />
    <!--<Policies>
      <RunAsPolicy CodePackageRef="Code" UserRef="AdminUser" EntryPointType="All" />
    </Policies>-->
  </ServiceManifestImport>
  <DefaultServices>
    <!-- The section below creates instances of service types, when an instance of this 
         application type is created. You can also create one or more instances of service type using the 
         ServiceFabric PowerShell module.
         
         The attribute ServiceTypeName below must match the name defined in the imported ServiceManifest.xml file. -->
    <Service Name="Traefik" ServicePackageActivationMode="ExclusiveProcess">
      <StatelessService ServiceTypeName="TraefikType" InstanceCount="[Traefik_InstanceCount]">
        <SingletonPartition />
      </StatelessService>
    </Service>

    <!-- UNCOMMENT TO ENABLE WATCHDOG AS DEFAULT SERVICE -->
    <!--
    <Service Name="Watchdog" ServicePackageActivationMode="ExclusiveProcess">
      <StatelessService ServiceTypeName="WatchdogType" InstanceCount="[Traefik_InstanceCount]">
        <SingletonPartition />
      </StatelessService>
    </Service>
    -->
  </DefaultServices>
</ApplicationManifest>

My cluster is secure and I've used the certificate which is securing the cluster to generate the .crt and .key files as described in the documentation.

When I publish through visual studio to the SF cluster, I'm still getting an error:
The EntryPoint traefik_linux-amd64 is not found.

The complete output is:

1>------ Build started: Project: Traefik, Configuration: Debug x64 ------
2>------ Publish started: Project: Traefik, Configuration: Debug x64 ------
2>Started executing script 'GetApplicationExistence'.
2>Finished executing script 'GetApplicationExistence'.
2>Time elapsed: 00:00:01.7797888
-------- Package started: Project: Traefik, Configuration: Debug x64 ------
Traefik -> C:\LinuxCluster\traefik-on-service-fabric\Traefik\pkg\Debug
-------- Package: Project: Traefik succeeded, Time elapsed: 00:00:00.7181757 --------
2>Started executing script 'Deploy-FabricApplication.ps1'.
2>powershell -NonInteractive -NoProfile -WindowStyle Hidden -ExecutionPolicy Bypass -Command ". 'C:\LinuxCluster\traefik-on-service-fabric\Traefik\Scripts\Deploy-FabricApplication.ps1' -ApplicationPackagePath 'C:\LinuxCluster\traefik-on-service-fabric\Traefik\pkg\Debug' -PublishProfileFile 'C:\LinuxCluster\traefik-on-service-fabric\Traefik\PublishProfiles\Cloud.xml' -DeployOnly:$false -ApplicationParameter:@{} -UnregisterUnusedApplicationVersionsAfterUpgrade $false -OverrideUpgradeBehavior 'None' -OverwriteBehavior 'SameAppTypeAndVersion' -SkipPackageValidation:$false -ErrorAction Stop"
2>Copying application to image store...
2>Upload to Image Store succeeded
2>Registering application type...
2>Register application type started. Use Get-ServiceFabricApplicationType to query for status.
2>Running Image Builder process ...
2>Downloading application package from 'TraefikType' ...
2>Downloaded 6/7 Files (85.7% complete)
2>Validating package and computing checksums ...
2>FABRIC_E_IMAGEBUILDER_VALIDATION_ERROR: The EntryPoint traefik_linux-amd64 is not found.
2>FileName: /mnt/sfroot/ImageBuilderProxy/AppType/088eb6bc-c45b-420f-8d5b-4f7a84e72b7c/TraefikPkg/ServiceManifest.xml
2>Registration of application type failed.
2>At C:\Program Files\Microsoft SDKs\Service 
2>Fabric\Tools\PSModule\ServiceFabricSDK\Publish-NewServiceFabricApplication.ps1:308 char:17
2>+                 throw "Registration of application type failed."
2>+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2>    + CategoryInfo          : OperationStopped: (Registration of application type failed.:String) [], RuntimeException
2>    + FullyQualifiedErrorId : Registration of application type failed.
2> 
2>Finished executing script 'Deploy-FabricApplication.ps1'.
2>Time elapsed: 00:03:05.8876450
2>The PowerShell script failed to execute.
========== Build: 1 succeeded or up-to-date, 0 failed, 0 skipped ==========
========== Publish: 0 succeeded, 1 failed, 0 skipped ==========

This is the file structure:

Could you maybe show what your ApplicationManifest.xml looks like?

Also, I have traefik working on windows SF cluster with routing working, I've written a powershell script to tag the services in the cluster and then route to then based on url path. If this is what you are looking for I could share the script with you?

javaadpatel07 · July 23, 2019, 12:28pm

I've gotten it working now, atleast it is healthy in SF, I still need to do the port mapping on the load balancer. Thank you so much for the help, the issue was firstly that I was using the wrong executable (windows one instead of linux) and then I had the extension .exe on the executable but this only applies to windows not linux.

Thanks again for the help.

solidcloudio · July 23, 2019, 12:55pm

No problem... Good luck!