I understand that I can have a policy for a user but how to I have rate limit for the API across all users?
For example I might have 10 req/sec rate limit per user but I do not want the gateway to do 1000 req/sec for a specific API no matter how many unique users make request. This will help to protect a backend or to protect the gateway itself.
This is currently not supported - you can rate limit on a user basis only. You could loop the service back on itself though, and have one API Definition point at another one (localhost), then inject a token into all inbound requests that sets an application-wide rate limit, then the second proxy will handle the application wide limit and the first the user-specific one. Not ideal but it will do what you require.