Extending Pumps in Tyk-Pump

1

I am currently trying to implement firehose into tyk-pump. Im having an issue where its not recognizing the new pump type. In the pumps/init.go I have added the following:
AvailablePumps["firehose"] = &FirehosePump{}

In pumps/pump.go I have added
case "firehose": return AvailablePumps["firehose"], nil

and i have created a new file called pumps/firehose.go with a New(), GetName(), Init() and WriteData() methods.

In the pump.conf ive added a “firehose” key wtih the name as firehose, and the respected meta data and its shooting me a [Sep 9 10:17:04] ERROR main: Pump load error (skipping): Not found
I can debug it, and see that “firehose” is whats being passed to the switch statement, but its skipping it and returning the error. I have no clue where to go from here with it. Is there something im missing?

2

Hi Eric, is your code available somewhere?

3

I can paste it in here. I havent forked it or anything yet, I was going to see if I could get this to work on my local, before I made a contribution.

4

I haven’t checked the Pump code for a while, but I would be happy to give it a quick build and see how’s going.

5

Sounds good. Thanks for looking into it

6 
package pumps

import "github.com/TykTechnologies/tyk-pump/logger"

var log = logger.GetLogger()
var AvailablePumps map[string]Pump

func init() {
	AvailablePumps = make(map[string]Pump)

	// Register all the storage handlers here
	AvailablePumps["dummy"] = &DummyPump{}
	AvailablePumps["mongo"] = &MongoPump{}
	AvailablePumps["mongo-pump-selective"] = &MongoSelectivePump{}
	AvailablePumps["mongo-pump-aggregate"] = &MongoAggregatePump{}
	AvailablePumps["csv"] = &CSVPump{}
	AvailablePumps["elasticsearch"] = &ElasticsearchPump{}
	AvailablePumps["influx"] = &InfluxPump{}
	AvailablePumps["statsd"] = &StatsdPump{}
	AvailablePumps["segment"] = &SegmentPump{}
	AvailablePumps["graylog"] = &GraylogPump{}
	AvailablePumps["firehose"] = &FirehosePump{}
}
7

Its like its skipping the firehose case for whatever reason in this code
package pumps

import (
	"errors"
)

type Pump interface {
	GetName() string
	New() Pump
	Init(interface{}) error
	WriteData([]interface{}) error
}

func GetPumpByName(name string) (Pump, error) {
	switch name {
	case "dummy":
		return AvailablePumps["dummy"], nil
	case "mongo":
		return AvailablePumps["mongo"], nil
	case "mongo-pump-selective":
		return AvailablePumps["mongo-pump-selective"], nil
	case "mongo-pump-aggregate":
		return AvailablePumps["mongo-pump-aggregate"], nil
	case "elasticsearch":
		return AvailablePumps["elasticsearch"], nil
	case "csv":
		return AvailablePumps["csv"], nil
	case "influx":
		return AvailablePumps["influx"], nil
	case "statsd":
		return AvailablePumps["statsd"], nil
	case "segment":
		return AvailablePumps["segment"], nil
	case "graylog":
		return AvailablePumps["graylog"], nil
	case "firehose":
		return AvailablePumps["firehose"], nil
	}

	return nil, errors.New("Not found")
}
8

I think your changes are fine, it could be something related to GOPATH, note that there are different Go packages in Tyk Pump, so one package, the main one is: github.com/TykTechnologies/tyk-pump.

The other one is github.com/TykTechnologies/tyk-pump/pumps, so I’m guessing that your changes don’t get build, and you’re still building the unmodified version of this package.

How’s your development environment set?

9

DANG THAT WAS IT! I <3 YOU. sorry lol

1 Like
10

Yes, consider doing a fork, so then you go get your fork, and you will be fine.

Best