Imported Google Group message. Original thread at: Redirecting to Google Groups Import Date: 2016-01-19 21:39:40 +0000.
Sender:Dmitry Nikolayev.
Date:Tuesday, 15 December 2015 14:19:39 UTC.
Hello,
We have a situation with lost redis data.
Redis was started with clean DB and after this all keys are not authorised on requesting tyk urls.
But I see all of this keys in dashboard.
As I understood, keys are stored in mongo and redis (added on creation of key).
Is there any way to sync keys into redis again?
Imported Google Group message.
Sender:Martin Buhr.
Date:Tuesday, 15 December 2015 19:27:27 UTC.
Hi Dmitry,
Keys are stored in Redis, if you deleted your Redis DB then you have lost your keys.
Key references (if you are not hashing keys) will be added to the developer record of your Portal developers.
You could extract the keys themselves (if they are unhashed), then use the Tyk REST API to manually add them back into Tyk as custom keys.
The keys you see in analytics are just that, they are analytics data, not linked to a real key.
Hope that helps - we really recommend making sure you snapshot Redis or enable some kind of persistence.
Cheers,
Martin
From: Dmitry Nikolayev [email protected]
Sent: Tuesday, December 15, 2015 14:19
Subject: Move keys from Mongo to Redis
To: Tyk Community Support [email protected]
–
You received this message because you are subscribed to the Google Groups “Tyk Community Support” group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web, visit https://groups.google.com/d/msgid/tyk-community-support/6037dfb2-59cf-4e45-91a2-8b5f266c9a8e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Imported Google Group message.
Sender:Dmitry Nikolayev.
Date:Tuesday, 15 December 2015 20:09:45 UTC.
Hey, Martin
Thnx for quick reply!
We are not using the hashed keys. It would be nice to have some option or utility inside tyk to do such restores.
If I have time, I will try to make a script… as I see inside redis - it is just keys with some not hard JSON inside.
The other question - we upgraded to 0.9.7… and it runs dashboard in hashed keys mode… even with hash option set to false inside conf file.
On first run it was ok, but after few restarts it started to work in hashed mode. (checked both: gateway and dashboard)
Can you help how to fix this? (data in redis and mongo is not hashed as I see now)
Imported Google Group message.
Sender:Martin Buhr.
Date:Tuesday, 15 December 2015 20:15:58 UTC.
Hi Dmitry,
You can’t switch between hashed and non hashed, those values will be written to analytics permanently as they were processed, if you switched you’ll end up with a mixed set.
Tyk runs in hashed mode by default, are you sure it’s disabled in both Tyk and Tyk dashboard? You might need to blitz your DB so it’s not a mixed key set.
Cheers,
Martin
-
show quoted text -
-
show quoted text -
To view this discussion on the web, visit https://groups.google.com/d/msgid/tyk-community-support/a2d351c3-ef33-4781-abb4-7c3cf1300e58%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Imported Google Group message.
Sender:Dmitry Nikolayev.
Date:Tuesday, 15 December 2015 20:22:29 UTC.
Just rechecked…
Both has:
“hash_keys”:false
Dropped all analytics events… restarted tyk - all events are saved without hashed key
But dashboard still in hashed mode… so no stats in portal…
Imported Google Group message.
Sender:Martin Buhr.
Date:Tuesday, 15 December 2015 20:26:03 UTC.
Hi Dmitry,
Analytics has nothing to do with hashed keys, it just records what’s coming through, if analytics aren’t showing up in the dashboard - check the month in the drop down and set it to 1, there’s a dashboard error where on December it thinks to make the maximum month 0, so stats for the day get a little funny.
Cheers,
Martin
-
show quoted text -
-
show quoted text -
To view this discussion on the web, visit https://groups.google.com/d/msgid/tyk-community-support/0438246a-af79-44b0-80dc-9df78df6747c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Imported Google Group message.
Sender:Dmitry Nikolayev.
Date:Tuesday, 15 December 2015 20:48:21 UTC.
I understand…
look, in 0.9.6.5 I used {{$apiData.APIKey}} in templates to show the KEY to customer via portal.
Now it looks as hashed (as I saw in 0.9.6.5 when i tried hashed mode).
And graph is not loaded in portal… (blank values)
Do you know how can i fix this?
Imported Google Group message.
Sender:Martin Buhr.
Date:Tuesday, 15 December 2015 21:29:45 UTC.
Hi Dmitry,
To fix the graph, see my last email - I think it’s just the graph not loading properly because the date parameter is set wrongly, just change the end date and click the blue icon to load the data again. As I said, if the data is in Kingo, there’s no reason for it not to show up (hashed or not), it’s a UI bug.
As for the template issue - if you created the key at some point when the dashboard was in hashed mode, then that key will forever be hashed, they are stored that way.
If, however, they key was created with hashed mode disabled (and you can list it in the dashboard keys section and see it without needing to do a key lookup), but it shows as hashed in the portal, that’s a bug we need to look at.
If your dashboard keys section does not list keys, but instead only shows a search view, then hashed mode is still enabled.
Tyk v1.9 and dashboard keep their configs in /opt/tyk-* not in /etc so make sure you are looking at the right conf files after an upgrade.
Let me know if any of the above apply.
Cheers,
Martjn
From: Dmitry Nikolayev [email protected]
Sent: Tuesday, December 15, 2015 20:48
Subject: Re: Move keys from Mongo to Redis
To: Tyk Community Support [email protected]
Cc: [email protected]
To view this discussion on the web, visit https://groups.google.com/d/msgid/tyk-community-support/eec875e0-bc51-4f32-817f-89098afd4a27%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Imported Google Group message.
Sender:Martin Buhr.
Date:Tuesday, 15 December 2015 21:58:25 UTC.
Hi Dmitry,
I just ran the latest v1.9 docker on a clean install, but changed it to use unhashed configs.
APIs loaded fine, keys were created unhashed, they show up in the key list (though there’s a wee bug there we need to address that’s unrelated).
The key, when accessed, loads fine in its unhashed configuration (no lookup). This means it is being stored unhashed in the DB
Using the API with the key worked too.
I didn’t get to test the portal, but since hashing is a centralised function it should behave the same throughout.
Just confirming that it’s behaving as expected, so there’s most likely a config issue here.
Cheers,
Martin
-
show quoted text -
-
show quoted text -
To view this discussion on the web, visit https://groups.google.com/d/msgid/tyk-community-support/eec875e0-bc51-4f32-817f-89098afd4a27%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Imported Google Group message.
Sender:Dmitry Nikolayev.
Date:Tuesday, 15 December 2015 22:12:42 UTC.
I think that probably someone played with options in our dev conf…
Just deleted that developer and recreated it - now seems everything is fine.
But it is wierd that everything in mongo/redis looked fine…
Also, Martin… Any news about improving portal?
Graphs inside dashboard are awesome as the tyk itself,… it would be nice if it is possible to show same quality graphs in portal export data as csv, for instance…
Imported Google Group message.
Sender:Martin Buhr.
Date:Tuesday, 15 December 2015 22:14:50 UTC.
Hi Dmitry,
Glad it’s sorted 
We’d love to improve the portal - mind expanding on some of your ideas?
Cheers,
Martin
-
show quoted text -
-
show quoted text -
To view this discussion on the web, visit https://groups.google.com/d/msgid/tyk-community-support/d0de6b5e-97a0-474e-a67c-ce82d8f355b1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Imported Google Group message.
Sender:Dmitry Nikolayev.
Date:Wednesday, 16 December 2015 15:12:28 UTC.
You have cool interface in dashboard… just copy per user stats in portal… same style…
And also there should be some way to export data… in csv/excel (probably csv is ok) for given period.
This is the greatest feature you can add now.
I tried to add graphs in templates, but it is like a pain in the ass… 
Would be nice if I can edit which graphs I want to see of users portal too…
P.s. one of the users after day of work became to work like with hashed key again… Code was not changed, only tyk-dashboard restarting because of templates editing.
Checked in mongo - it has saved subscription in hashed mode by some reason now…
Created another user- everything is fine…
Whatever, I’ll try to keep checking for this
Imported Google Group message.
Sender:Dmitry Nikolayev.
Date:Monday, 4 January 2016 13:09:25 UTC.
Hey, Martin
I thought that we may have some free time of our frontend developer.
Maybe We can help to make interface of portal (for developer) looking better?
I mean, if you share some docs about graphs/apis used in analytics - we can apply the same to developer level?
I think, it may be a good contribution to support all your work?
Imported Google Group message.
Sender:Martin Buhr.
Date:Monday, 4 January 2016 13:38:40 UTC.
Hi Dmitry,
The portal is part of Tyk Analytics, which is a closed source code base. However we’d be more than happy to take recommendations, and it would be of great benefit to the community if you’ve got improved / modified portal templates that are better / personalised from ours. We can’t merge them into our distro, but users can certainly download and use them if they want an alternative skin (and we might borrow some ideas).
One way you could make this available to the community is to share your portal templates as a github repository so others can download and use them?
Regarding data access, the endpoint API that powers the data on the developer interface is restricted to a specific key (not an aggregate), however it can handle all of the parameters that the dashboard data APIs can.
The analytics API is not documented, but you can quite easily see how it works by watching what the dashboard GUI is doing on the network console, it’s pretty formulaic, the endpoint is:
/activity/me/:orgId/:keyId/:sday/:smonth/:syear/:eday/:emonth/:eyear?res={hour|day|month|year}[&sort=1&sort_by={DBFieldName}]
This will not group data though, all the grouping actions (e.g. get error breakdown or endpoint activity grouped and processed) is not built into the endpoint, it would need a new endpoint to be added or the existing one to be extended.
The data caller that handles the charts in the current templates also does some sorting and data wrangling client-side (not the cleanest), so there may be work needed there too.
I love the idea, and happy to help where we can.
Cheers,
Martin