Identity-broker: LDAP+token for arbitrary api endpoint (no dashboard)

I am a newbie with Tyk. Trying to make a demo project with set of products (i use docker to make setup more visualised for presentation, so it’s easy to try for others):

  • tykio/tyk-gateway:latest
  • tykio/tyk-pump-docker-pub:latest
  • tyk-identity-broker:latest

Seems that products are very good, but i face some lack of just working examples (docker-compose files, configs) in addition to documentation, which contain too long texts. Working examples could make start process many times faster as well as could help to share best practices provided by vendor.

My use case:
I want to authorise user against of LDAP, generate a temporary token and forward authorised request to secured page “http://${BASE_URL}/?success=true”, where
BASE_URL=http://localhost:8080

I do not use dashboard so most probably because of this I face lack of good and working examples, and do not plan to use it future, so face now problems with “IdentityHandlerConfig”, because most of examples have only dashboard identity handler.

My profiles.json:
[
{
“ActionType”: “GenerateTemporaryAuthToken”,
“ID”: “1.0”,
“IdentityHandlerConfig”: {
“TokenAuth”: {
“BaseAPIID”: “app”,
“RedirectURI”: “${BASE_URL}/?success=true”
}
},
“MatchedPolicyID”: “it-was-ldap-auth”,
“OrgID”: “org-id”,
“ProviderConfig”: {
“FailureRedirect”: “${BASE_URL}/?fail=true”,
“LDAPAttributes”: [],
“LDAPPort”: “389”,
“LDAPServer”: “ldap.forumsys.com”,
“LDAPUserDN”: “cn=read-only-admin,dc=example,dc=com”
},
“ProviderName”: “ADProvider”,
“ReturnURL”: “${BASE_URL}/?success=true”,
“Type”: “redirect”
}
]

my tib.conf:
{
“HttpServerOptions”: {
“UseSSL”: false,
“CertFile”: “./certs/server.pem”,
“KeyFile”: “./certs/server.key”
},
“BackEnd”: {
“Name”: “in_memory”,
“IdentityBackendSettings”: {
“Hosts” : {
“redis”: “6379”
},
“Password”: “”,
“Database”: 0,
“EnableCluster”: false,
“MaxIdle”: 1000,
“MaxActive”: 2000,
“UseSSL”: false,
“SSLInsecureSkipVerify”: false
}
},
“TykAPISettings”: {
“GatewayConfig”: {
“Endpoint”: “${BASE_URL}”,
“Port”: “8080”
}
}
}

and it fails on the token generating after authorisation through LDAP:

tyk-identity    | time="2020-05-14T18:04:18Z" level=info msg="Search: starting..." prefix="AD AUTH"
tyk-identity    | time="2020-05-14T18:04:18Z" level=info msg="Running LDAP search" DN="cn=read-only-admin,dc=example,dc=com" Filter="(objectclass=*)" prefix="AD AUTH"
tyk-identity    | time="2020-05-14T18:04:18Z" level=warning msg="User email not found, generating from username" prefix="AD AUTH"
tyk-identity    | time="2020-05-14T18:04:18Z" level=info msg="Starting Token Flow..." prefix="TYK ID HANDLER"
tyk-identity    | time="2020-05-14T18:04:18Z" level=error msg="Failed to create request" error="parse :/api/keys: missing protocol scheme" prefix="TYK_API"
tyk-identity    | 2020/05/14 18:04:18 http: panic serving 172.25.0.1:58932: runtime error: invalid memory address or nil pointer dereference
tyk-identity    | goroutine 69 [running]:
tyk-identity    | net/http.(*conn).serve.func1(0xc000188c80)
tyk-identity    | 	/usr/local/go/src/net/http/server.go:1769 +0x139
tyk-identity    | panic(0x88b380, 0xcff8f0)
tyk-identity    | 	/usr/local/go/src/runtime/panic.go:522 +0x1b5
tyk-identity    | github.com/TykTechnologies/tyk-identity-broker/tyk-api.(*TykAPI).DispatchDashboard(0xd0d900, 0x90a0ad, 0x9, 0x908d0d, 0x4, 0x0, 0x0, 0x9aeb20, 0xc000146600, 0x17, ...)
tyk-identity    | 	/src/github.com/TykTechnologies/tyk-identity-broker/tyk-api/tyk_api.go:145 +0x2f9
tyk-identity    | github.com/TykTechnologies/tyk-identity-broker/tyk-api.(*TykAPI).DispatchAndDecode(0xd0d900, 0x90a0ad, 0x9, 0x908d0d, 0x4, 0x908da5, 0x4, 0x8495e0, 0xc00012eea0, 0x0, ...)
tyk-identity    | 	/src/github.com/TykTechnologies/tyk-identity-broker/tyk-api/tyk_api.go:268 +0x6a0
tyk-identity    | github.com/TykTechnologies/tyk-identity-broker/tyk-api.(*TykAPI).RequestStandardToken(0xd0d900, 0xc000177bf4, 0x6, 0xc000177c00, 0x10, 0xc000177c79, 0x7, 0x0, 0x0, 0xe10, ...)
tyk-identity    | 	/src/github.com/TykTechnologies/tyk-identity-broker/tyk-api/tyk_api.go:513 +0x944
tyk-identity    | github.com/TykTechnologies/tyk-identity-broker/tap/identity-handlers.(*TykIdentityHandler).CompleteIdentityActionForTokenAuth(0xc000242000, 0x9b7a60, 0xc00023c000, 0xc0001fe300, 0x8f7120, 0xc000288000, 0xc000177bf0, 0x3, 0xc000177bf4, 0x6, ...)
tyk-identity    | 	/src/github.com/TykTechnologies/tyk-identity-broker/tap/identity-handlers/tyk_handler.go:429 +0x3b8
tyk-identity    | github.com/TykTechnologies/tyk-identity-broker/tap/identity-handlers.(*TykIdentityHandler).CompleteIdentityAction(0xc000242000, 0x9b7a60, 0xc00023c000, 0xc0001fe300, 0x8f7120, 0xc000288000, 0xc000177bf0, 0x3, 0xc000177bf4, 0x6, ...)
tyk-identity    | 	/src/github.com/TykTechnologies/tyk-identity-broker/tap/identity-handlers/tyk_handler.go:486 +0x146
tyk-identity    | github.com/TykTechnologies/tyk-identity-broker/providers.(*ADProvider).Handle(0xc000240000, 0x9b7a60, 0xc00023c000, 0xc0001fe300)
tyk-identity    | 	/src/github.com/TykTechnologies/tyk-identity-broker/providers/active_directory.go:317 +0xcb7
tyk-identity    | main.HandleAuth(0x9b7a60, 0xc00023c000, 0xc0001fe300)
tyk-identity    | 	/src/github.com/TykTechnologies/tyk-identity-broker/http_handlers.go:138 +0x83
tyk-identity    | net/http.HandlerFunc.ServeHTTP(0x927550, 0x9b7a60, 0xc00023c000, 0xc0001fe300)
tyk-identity    | 	/usr/local/go/src/net/http/server.go:1995 +0x44
tyk-identity    | github.com/TykTechnologies/tyk-identity-broker/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc000178410, 0x9b7a60, 0xc00023c000, 0xc0001fe300)
tyk-identity    | 	/src/github.com/TykTechnologies/tyk-identity-broker/vendor/github.com/gorilla/mux/mux.go:114 +0xdb
tyk-identity    | net/http.serverHandler.ServeHTTP(0xc00019c680, 0x9b7a60, 0xc00023c000, 0xc0001fe100)
tyk-identity    | 	/usr/local/go/src/net/http/server.go:2774 +0xa8
tyk-identity    | net/http.(*conn).serve(0xc000188c80, 0x9b89e0, 0xc00017c2c0)
tyk-identity    | 	/usr/local/go/src/net/http/server.go:1878 +0x851
tyk-identity    | created by net/http.(*Server).Serve
tyk-identity    | 	/usr/local/go/src/net/http/server.go:2884 +0x2f4

Any ideas how to reconfigure IdentityHandlerConfig to register token and redirect it to the “${BASE_URL}/?success=true” or respond client with json token ?

After short investigation, i see that failure has gone when I append “DashboardConfig” into tib.conf with ${DASHBOARD_URL}=http://dashboard.local (sry for this variable, but policy that new user can add only 2 urls is quite annoying and I had to replace everywhere url with this variable, even in the log message):

"DashboardConfig": {
       "Endpoint": "${DASHBOARD_URL}",
       "Port": "3000",
       "AdminSecret": "352d20ee67be67f6340b4c0605b044b7"
   }

so that i just have error:
tyk-identity | time=“2020-05-14T20:16:12Z” level=warning msg=“No expiry found - defaulting to 3600 seconds” prefix=“TYK ID HANDLER”
tyk-identity | time=“2020-05-14T20:16:12Z” level=info msg=“Search: starting…” prefix=“AD AUTH”
tyk-identity | time=“2020-05-14T20:16:12Z” level=info msg=“Running LDAP search” DN=“cn=read-only-admin,dc=example,dc=com” Filter=“(objectclass=*)” prefix=“AD AUTH”
tyk-identity | time=“2020-05-14T20:16:12Z” level=warning msg=“User email not found, generating from username” prefix=“AD AUTH”
tyk-identity | time=“2020-05-14T20:16:12Z” level=info msg=“Starting Token Flow…” prefix=“TYK ID HANDLER”
tyk-identity | time=“2020-05-14T20:16:22Z” level=info msg=error dispatchErr=“Post ${DASHBOARD_URL}/api/keys: dial tcp: lookup dashboard.local: Temporary failure in name resolution” prefix=“TYK_API” retCode=500
tyk-identity | time=“2020-05-14T20:16:22Z” level=error msg=“Failed to generate Auth token” error=“Post ${DASHBOARD_URL}/api/keys: dial tcp: lookup dashboard.local: Temporary failure in name resolution” prefix=“TYK ID HANDLER”

which is logical, because i do not have dashboard and do not plan to use it right now, but new keys it tries to register in dashboard service, which is relatively unexpected because desired flow was: //tyk.io/docs/tyk-identity-broker/

for ActionType: GenerateTemporaryAuthToken I understood that identityProker generates a standard access token for the logged in user, and then can be delivered as a redirect fragment OR as a direct API response (JSON) without usage of dashboard api endpoint “/api/keys”

and after another change (already in the gateway) redirect happens properly, but token is empty.
current configuration of the gateway application (apps/app.json):

{
  "name": "app",
  "api_id": "app",
  "use_keyless": true,
  "version_data": {
    "not_versioned": true,
    "default_version": "",
    "versions": {
      "default": {
        "name": "default",
        "use_extended_paths": true
      }
    }
  },
  "proxy": {
    "listen_path": "/api/",    //<-------- initialy here was "/" so that api/keys was not possible to access
    "target_url": "http://app.localhost/index.php",
    "strip_listen_path": true
  },
  "active": true,
  "tag_headers": [],
  "strip_auth_data": false
}

as soon as i have changed “listen_path” from “/” to “/api/”, i also replaced accordingly configuration of the identity broker, so that “profiles.json” looks like:

[
  {
    "ActionType": "GenerateTemporaryAuthToken",
    "ID": "1.0",
    "MatchedPolicyID": "it-was-ldap-auth",
    "OrgID": "org-id",
    "ProviderConfig": {
      "FailureRedirect": "http://localhost:8080/api/?fail=true",
      "LDAPAttributes": [],
      "LDAPPort": "389",
      "LDAPServer": "ldap.forumsys.com",
      "LDAPUserDN": "cn=read-only-admin,dc=example,dc=com"
    },
    "ProviderName": "ADProvider",
    "ReturnURL": "http://localhost:8080/api/?success=true",
    "Type": "redirect"
  }
]

and in the tib.conf, instead of non existning dasbboard url, i put gateway endpoint, so it looks like

"DashboardConfig": {
       "Endpoint": "http://tyk-gateway",
       "Port": "8080",
       "AdminSecret": "352d20ee67be67f6340b4c0605b044b7"
   }

now, after I call LDAP login, i see in the logs (seems to be success):

    tyk-identity    | time="2020-05-14T22:07:14Z" level=info msg="Search: starting..." prefix="AD AUTH"
    tyk-identity    | time="2020-05-14T22:07:14Z" level=info msg="Running LDAP search" DN="cn=read-only-admin,dc=example,dc=com" Filter="(objectclass=*)" prefix="AD AUTH"
    tyk-identity    | time="2020-05-14T22:07:14Z" level=warning msg="User email not found, generating from username" prefix="AD AUTH"
    tyk-identity    | time="2020-05-14T22:07:14Z" level=info msg="Starting Token Flow..." prefix="TYK ID HANDLER"
    demo-app        | 172.25.0.6 - - [14/May/2020:22:07:14 +0000] "POST /index.php/keys HTTP/1.1" 200 22557 "-" "Go-http-client/1.1"
    tyk-identity    | time="2020-05-14T22:07:14Z" level=info msg="--> Running auth redirect..." prefix="TYK ID HANDLER"
    demo-app        | 172.25.0.6 - - [14/May/2020:22:07:14 +0000] "GET /index.php?success=true HTTP/1.1" 200 22930 "http://localhost:8080/api/?success=true" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:76.0) Gecko/20100101 Firefox/76.0"
    tyk-datapump    | time="2020-05-14T22:07:15Z" level=info msg="Writing 2 records" prefix=elasticsearch-pump

but token still empty, after login as desired in config it does redirect to http://localhost:8080/api/?success=true#token=
but token is empty for now and a little bit confusing in the log looks this line record “POST /index.php/keys HTTP/1.1”. In headers also no any information to client about current token. Is there any way to obtain this token? As alternative solution of course i can delegate tockens genertion to application ifself, but i wanted to use as much as possible from tyk identity and gateway.

And my latest experiment shows that i have authorised, but token can not be generated.
So maybe you can point whats wrong with configuration.

Authorisation through open LDAP is successful, but i get empty key id afterwards:
http://localhost:3010/auth/1.0/ldap

Request URL:http://localhost:3010/auth/1.0/ldap
HTTP/1.1 200 OK
Content-Type: application/json
Date: Fri, 15 May 2020 08:22:29 GMT
Content-Length: 13

{"key_id":""}

and in the log:

tyk-identity    | time="2020-05-15T08:25:57Z" level=info msg="Search: starting..." prefix="AD AUTH"
tyk-identity    | time="2020-05-15T08:25:57Z" level=info msg="Running LDAP search" DN="cn=read-only-admin,dc=example,dc=com" Filter="(objectclass=*)" prefix="AD AUTH"
tyk-identity    | time="2020-05-15T08:25:58Z" level=warning msg="User email not found, generating from username" prefix="AD AUTH"
tyk-identity    | time="2020-05-15T08:25:58Z" level=info msg="Starting Token Flow..." prefix="TYK ID HANDLER"
demo-app        | [APP][debug] Request headers:
demo-app        | array (
demo-app        |   'Host' => 'app.localhost',
demo-app        |   'User-Agent' => 'Go-http-client/1.1',
demo-app        |   'Content-Length' => '673',
demo-app        |   'Accept-Encoding' => 'gzip',
demo-app        |   'Authorization' => '',
demo-app        |   'X-Forwarded-For' => '172.25.0.4',
demo-app        | )[APP][debug] Request body:
demo-app        | array (
demo-app        | )172.25.0.6 - - [15/May/2020:08:25:58 +0000] "POST /index.php/keys HTTP/1.1" 200 22561 "-" "Go-http-client/1.1"
tyk-identity    | time="2020-05-15T08:25:58Z" level=info msg="--> No redirect, returning token..." prefix="TYK ID HANDLER"
tyk-datapump    | time="2020-05-15T08:25:58Z" level=info msg="Writing 1 records" prefix=elasticsearch-pump

all seems to be ok, except for empty token in response and empty ‘Authorization’ => ‘’, header, when it hits application.

Identity Broker configs
“profiles.json”:

[
  {
    "ActionType": "GenerateTemporaryAuthToken",
    "ID": "1.0",
    "MatchedPolicyID": "it-was-ldap-auth",
    "OrgID": "org-id",
    "ProviderConfig": {
      "FailureRedirect": "http://localhost:8080/api/?fail=true",
      "LDAPAttributes": [],
      "LDAPPort": "389",
      "LDAPServer": "ldap.forumsys.com",
      "LDAPUserDN": "cn=read-only-admin,dc=example,dc=com"
    },
    "ProviderName": "ADProvider",
    "ReturnURL": "",
    "Type": "passthrough"
  }
]

tib.conf

{
 "HttpServerOptions": {
   "UseSSL": false,
   "CertFile": "./certs/server.pem",
   "KeyFile": "./certs/server.key"
 },
 "BackEnd": {
   "Name": "in_memory",
   "IdentityBackendSettings": {
     "Hosts" : {
         "redis": "6379"
     },
     "Password": "",
     "Database": 0,
     "EnableCluster": false,
     "MaxIdle": 1000,
     "MaxActive": 2000,
     "UseSSL": false,
     "SSLInsecureSkipVerify": false
   }
 },
 "TykAPISettings": {
   "GatewayConfig": {
     "Endpoint": "http://tyk-gateway",
     "Port": "8080",
     "AdminSecret": "352d20ee67be67f6340b4c0605b044b7"
   },
   "DashboardConfig": {
       "Endpoint": "http://tyk-gateway",
       "Port": "8080",
       "AdminSecret": "352d20ee67be67f6340b4c0605b044b7"
   }
 }
}

Gateway configs

apps/app.json:

{
  "name": "app",
  "api_id": "app",
  "use_keyless": true,
  "version_data": {
    "not_versioned": true,
    "default_version": "",
    "versions": {
      "default": {
        "name": "default",
        "use_extended_paths": true
      }
    }
  },
  "openid_options": {
    "providers": [],
    "segregate_by_client": false
  },
  "proxy": {
    "listen_path": "/api/",
    "target_url": "http://app.localhost/index.php",
    "strip_listen_path": true
  },
  "active": true,
  "tag_headers": [],
  "allowed_ips": [],
  "strip_auth_data": false
}

tyk.conf

{
  "listen_port": 8080,
  "secret": "352d20ee67be67f6340b4c0605b044b7",
  "template_path": "/opt/tyk-gateway/templates",
  "tyk_js_path": "/opt/tyk-gateway/js/tyk.js",
  "middleware_path": "/opt/tyk-gateway/middleware",
  "use_db_app_configs": false,
  "app_path": "/opt/tyk-gateway/apps/",
  "storage": {
    "type": "redis",
    "host": "redis",
    "port": 6379,
    "username": "",
    "password": "",
    "database": 0,
    "optimisation_max_idle": 2000,
    "optimisation_max_active": 4000
  },
  "health_check": {
    "enable_health_checks": true,
    "health_check_value_timeouts": 60
  },
  "optimisations_use_async_session_write": true,
  "enable_non_transactional_rate_limiter": true,
  "enable_sentinel_rate_limiter": false,
  "enable_redis_rolling_limiter": false,
  "enable_hashed_keys_listing": true,
  "allow_master_keys": false,
  "policies": {
    "policy_source": "file",
    "policy_record_name": "/opt/tyk-gateway/policies/policies.json"
  },
  "hash_keys": true,
  "close_connections": false,
  "http_server_options": {
    "enable_websockets": true
  },
  "allow_insecure_configs": true,
  "coprocess_options": {
    "enable_coprocess": true,
    "coprocess_grpc_server": ""
  },
  "enable_bundle_downloader": true,
  "bundle_base_url": "",
  "global_session_lifetime": 100,
  "force_global_session_lifetime": false,
  "max_idle_connections_per_host": 500
}