How to configure a Wire deployment for Multi-Ingress¶
What is a Multi-ingress setup?¶
Multi-domain single backend (also known as "multi-ingress" internally) is a setup to obfuscate the relationship of clients/users to a specific backend. To accomplish this, the clients (or groups of them) connect to domains that are provided by additional Kubernetes ingresses. I.e. one backend is made available under many domains.
How is multi-ingress useful?¶
This is useful to obfuscate the relationship of clients to each other, as an attacker on TCP/IP-level could only see domains and IPs that do not obviously relate to each other. Each of these backend domains represents a virtual backend.
To further obfuscate the fact that these multiple public IP addresses and domain names are tied to a single Wire backend, it would be desirable to extend these public IP addresses out away from the physical location where the services are being hosted. This can be achieved using common network routing, protocols, and techniques.
flowchart LR
%% Public side
I1((Internet))
I2((Internet))
I3((Internet))
R1["red.example.com 1.1.1.1"]
R2["green.example.org 2.2.2.2"]
R3["blue.example.net 3.3.3.3"]
I1 --> R1
I2 --> R2
I3 --> R3
WB["Wire Backend<br/>(1.2.3.4)<br/>single physical deployment"]
R1 --> WB
R2 --> WB
R3 --> WB
%% Color coding based on domains
classDef red fill:#ffebee,stroke:#c62828,stroke-width:2px,color:#c62828
classDef green fill:#e8f5e8,stroke:#388e3c,stroke-width:2px,color:#2e7d32
classDef blue fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#1565c0
classDef yellow fill:#fff9c4,stroke:#f57f17,stroke-width:2px,color:#4a4a4a
%% Apply colors according to domain
class R1 red
class R2 green
class R3 blue
class WB yellowNote: These backend domains are DNS domains only, not to be confused with the term "backend domain" used for federation (see Federation). In single-ingress setups, the backend DNS domain and federation backend domain are usually the same, but this is not true for multi-ingress setups. Multi-ingress and federation are mutually exclusive: federation must be disabled on a multi-ingress backend.
Requirements¶
- A Wire backend installation (> 5.14 to enable domain specific deeplinks, 5.37.0 for multi-ingress SSO)
- DNS records for domains to be used with all the required sub-domains. In the examples below, we demonstrate with 3 domains (
green.example.org,red.example.com,blue.example.net), but this setup can be extended to as many domains as needed:green.example.org(default domain)red.example.com(additional domain)blue.example.net(additional domain)- ... (add more domains as required)
- Load Balancers for each domain to forward the domain specific traffic to the single Wire-backend
- In order to further remove the connection between a domain configured to a wire-backend, the domain can have DNS records pointing to separate VPS (virtual private server) or network devices which can proxy the traffic for the backend. To have an effective proxy which can protect revealing the wire-backend details for other domains, we recommend setting up a HTTPS proxy at the VPS.
- Federation must be disabled — multi-ingress is not compatible with federation.
Instructions¶
1. Install Wire Backend¶
Install Wire Backend as per our guidelines. Wire in a Box (demo) solution can also be used for multi ingress.
For the multi-ingress setup, a default domain is required during the initial deployment. We have chosen to deploy the backend with green.example.org as the default domain. This domain serves the same purpose as the single domain used in traditional single-ingress setups - it is the primary backend domain that handles the core Wire services configuration.
Deploy wire-server with the following domain green.example.org as the main domain or default domain. Post deploying, make sure you have deployed wire and everything works at https://green.example.org.
Once the backend is operational with the default domain, additional domains can be added through the multi-ingress configuration steps that follow.
2. Components to re-configure for multi domain awareness¶
The following components need to be re-configured for multi domain awareness:
- nginz (deeplinks for multi-ingress)
- cannon
- cargohold
- galley
- spar
- webapp
- account-pages
- nginx-ingress-services
Instructions for required changes in wire-server values¶
Wire-server backend values can be found at: https://github.com/wireapp/wire-server-deploy/blob/master/values/wire-server/prod-values.example.yaml. Multi-ingress configuration settings are discussed in detail in the Multi-ingress setup section of the developers' config options reference.
Apart from already configured values for domain green.example.org, the following changes will be required:
Brig (optional)¶
This complements the FEATURE_ENABLE_ACCOUNT_REGISTRATION: false setting for the webapp (see Instructions for required changes in webapp values). setRestrictUserCreation prohibits public account creation via API. Creating users via SCIM or team invitations still works.
N.B. team owners are users as well. So, initial team owners must be created before this setting is activated or these users need to be created via internal endpoints.
Galley¶
settings.multiIngress and settings.conversationCodeURI are mutually exclusive. Cargohold¶
There shouldn't be any s3DownloadEndpoint, all endpoints should be under multiIngress.
Cannon¶
Nginz¶
Spar¶
To enable SSO via SAML in a multi-ingress setup, spar must act as a separate SAML Service Provider (SP) for each domain. This replaces the single-domain appUri/ssoUri/contacts values with a domainConfigs map keyed by the nginz-https.<domain> host (same key style as cargohold's aws.multiIngress above). scimBaseUri also becomes required, since it can no longer be derived from a single ssoUri:
idpCertFingerprintAllowlist is required for multi-ingress setups using SSO (spar refuses to create or update any IdP without it). It's a list of SHA-1 fingerprints, one per IdP signing certificate used across all domains — not a per-domain setting.
To get the fingerprint, download the IdP's SAML metadata for that domain. The download URL depends on the IdP; for example:
- Keycloak:
<keycloak-root>/realms/<realm>/protocol/saml/descriptor(see Keycloak: SAML v2.0 Identity Providers) - Microsoft Entra ID:
https://login.microsoftonline.com/<tenant-id>/federationmetadata/2007-06/federationmetadata.xml?appid=<application-id>(see Microsoft Entra federation metadata) - AD FS:
https://<adfs-fqdn>/federationmetadata/2007-06/federationmetadata.xml(see Microsoft: AD FS endpoints reference) - Okta: no fixed URL pattern — open your SAML app, go to the Sign On tab, and under SAML Signing Certificates select View IdP metadata to get the URL (see Okta: How to download Okta identity provider metadata and SAML signing certificates)
Once downloaded, extract the signing certificate from the metadata. Its base64 content is often wrapped across multiple lines in the file, so strip newlines before matching it:
Some IdPs publish more than one signing certificate at once (e.g. during a certificate rotation), in which case the grep above prints more than one match and the pipeline needs to run once per match instead:
Repeat for each IdP and add every resulting fingerprint to the allowlist.
Register each domain's IdP through that domain, not through the default one: call POST <api-version>/identity-providers via nginz-https.red.example.com for the IdP serving red.example.com, via nginz-https.blue.example.net for blue.example.net, and so on. Wire-server binds the new IdP to whichever domain you called it through. Only one IdP per domain/team is allowed — creating a second for the same domain and team fails.
Setting spar.config.enableIdPByEmailDiscovery: true enables the /sso/get-by-email endpoint, which looks up the right SSO code (IdP ID) for an activated SSO user's email on that domain (the user can be SCIM- or auto-provisioned). It's optional but highly recommended for multi-ingress SSO setups as otherwise users (clients) would have to manually provide one IdP code per domain.
Security: with multiple IdPs configured for one team, spar may migrate a user's SSO identity between IdPs based on a matching email NameID (see Multi-ingress cross-IdP SSO (fallback)). Email NameIDs must be unique and consistent across every IdP configured for the team — a collision can log a user into another user's account, and compromising one IdP compromises the team across all its domains.
See Spar and SAML IdPs for full reference.
Deploy wire-server chart¶
After making the above changes in values/wire-server/values.yaml, the wire-server helm chart should be re-deployed as:
Instructions for required changes in webapp values¶
Webapp values can be found at: https://github.com/wireapp/wire-server-deploy/blob/master/values/webapp/prod-values.example.yaml
Override the whole file with following:
Also ensure that the above environment variables are in sync with https://github.com/wireapp/wire-server-deploy/blob/master/values/webapp/prod-values.example.yaml in terms of names.
FEATURE_ENABLE_ACCOUNT_REGISTRATION would allow public account registration for everyone. This is usually not desirable for on-prems. Also, this feature depends on the team settings app, which isn't capable of being used via multiple ingresses, yet. (It's fine to use it to manage teams via the main-ingress, though.)
Deploy webapp helm chart¶
Re-deploy the webapp helm chart as following:
Instructions for required changes in account-pages values¶
Account-pages values can be found in the wire-account chart.
Unlike webapp, account-pages has no backendWebsocket/backendDomain env vars and no federation flag — only backendRest and appHost are consumed. [[hostname]] is a literal placeholder string, resolved server-side at request time (based on the incoming request's hostname) when ENABLE_DYNAMIC_HOSTNAME is true — same mechanism used by webapp.
Override the whole file with following:
Deploy account-pages helm chart¶
Re-deploy the account-pages helm chart as following:
Instructions for required changes in nginx-ingress-services values¶
The nginx-ingress-service chart should be deployed multiple times - once for each domain (default and additional domains). Each deployment uses unique names, distinct values files, and separate TLS certificates.
Note on Certificate Management: For simplicity in this guide, we demonstrate using Let's Encrypt to obtain TLS certificates. Initially, all DNS records for all domains should point to the ingress. If you prefer to use your own certificates instead of Let's Encrypt, refer to Bring your own certificates for detailed instructions on how to provide and manage your own certificates.
Initial deployment for default domain¶
The nginx-ingress-service chart for the default domain green.example.org should have been deployed before making the next steps. If not already done, refer to the initial deployment documentation.
Deploy additional domains with unique identities¶
For each additional domain (e.g., red.example.com, blue.example.net), you must deploy the nginx-ingress-service chart again with:
- Unique release names (e.g.,
nginx-ingress-services-red,nginx-ingress-services-blue) - Domain-specific values files with distinct configurations
- Separate TLS certificates (e.g.,
values/nginx-ingress-services/red-key.pem,values/nginx-ingress-services/red-cert.pem)
Prepare values for red domain¶
Prepare a unique helm chart values file for domain red.example.com as values/nginx-ingress-services/red-values.yaml:
Deploy red domain chart¶
Deploy this chart as following:
Prepare values for blue domain¶
Continue the process of deploying the helm charts for other domains, for example, we will do it for blue.example.net.
Create values/nginx-ingress-services/blue-values.yaml:
Deploy blue domain chart¶
It will be deployed as:
Patch the CSP (Content security policy) for each multi-ingress domain¶
Note: If you are on backend version 5.37 or higher then CSP configuration can be directly configured in the helm chart nginx-ingress-services at .cspExtraConnectSrc.
When the Webapp is used with multi-ingress calling, the ingress CSP configuration must explicitly allow the corresponding Calling endpoint (SFT_DOMAIN). This update ensures that the browser accepts resources and connections required for calling functionality. The same need to be repeated for each multi-ingress domain. To find out which SFT_DOMAIN can be used, continue reading the Calling section.
Verify the deployment¶
At the end of the process, the nginx should be configured to handle the traffic on all the different domains, try accessing the domains for deeplink and webapp respectively as:
- https://nginz-https.green.example.org/deeplink.json - https://webapp.green.example.org
- https://nginz-https.red.example.com/deeplink.json - https://webapp.red.example.com
- https://nginz-https.blue.example.net/deeplink.json - https://webapp.blue.example.net
Calling¶
For calling, Wire is dependent on two components: coturn and SFTD, as explained in overview. Since routing for calling traffic (UDP stream) works differently than HTTPS traffic for wire-backend, the calling components cannot be configured to support multiple network paths or separate endpoints for different domains. In other words, you cannot have different coturn or SFTD endpoints for each domain - all calling traffic must route through a single calling infrastructure.
As calling traffic can't be obfuscated in the same manner (i.e. using multiple domains), it is recommended to point all calling services to one publicly known source i.e. Wire cloud calling services (Managed by Wire). The Wire cloud calling service can be configured by checking in with Wire team here.
graph TB
CALLING["Shared Calling Infrastructure (Wire Cloud ☎️ Coturn & SFTD)"]
U1["🧑 Alice red.example.com"]
U2["🧑 Bob green.example.org"]
WB["Single Wire Backend"]
U1 -->|HTTPS messaging| WB
U1 -->|UDP calling| CALLING
U2 -->|HTTPS messaging| WB
U2 -->|UDP calling| CALLING
%% Color coding based on domain
classDef red fill:#ffebee,stroke:#c62828,stroke-width:2px,color:#c62828
classDef green fill:#e8f5e8,stroke:#388e3c,stroke-width:2px,color:#2e7d32
classDef yellow fill:#fff9c4,stroke:#f57f17,stroke-width:2px,color:#4a4a4a
classDef orange fill:#fff3e0,stroke:#ef6c00,stroke-width:2px,color:#e65100
%% Apply colors according to domain and function
class U1 red
class U2 green
class WB yellow
class CALLING orangeNote: This diagram is simplified for 2 domains. The same principle applies for any number of additional domains.
How to setup VPS for each domain between a client and wire-backend¶
Note: This diagram is simplified for 2 domains. The same principle applies for any number of additional domains.
We are going to use haproxy for setting a https proxy using a domain for the wire-backend. Any Ubuntu VPS can work for the proxy requirement. We need to ensure that DNS A records exists for the domain *.red.example.com (for all the sub-domains) and points to the VPS.
Note on Certificate Management: For simplicity in this guide, we demonstrate using Let's Encrypt to obtain TLS certificates for the VPS proxy. However, you can also bring and use your own certificates instead.
To set up the VPS with HAproxy and Let's Encrypt certs, follow the commands here:
Once the certificate is ready, store it at /etc/ssl/private/example.pem.
Use the following HAproxy config to receive traffic for domain red.example.com and forward it to wire-backend at (eg. 1.2.3.4):
Now restart the haproxy service using the command:
Now to confirm the behavior of the proxy, we can try to use the curl command:
Response:
And when asked for the information about the other domains, it should throw 403 Forbidden:
Response: