gen alpha

A library for bringing generics-like functionality to Go

gen is an attempt to bring some generics-like functionality to Go, with inspiration from C#’s Linq, JavaScript’s Array methods and the underscore library. Operations include filtering, grouping, sorting and more.

The pattern is to pass func’s as you would pass lambdas in Linq or functions in JavaScript.

Concepts

gen generates code for your types, at development time, using the command line. gen is not an import; the generated source becomes part of your project and takes no external dependencies.

You specify the types for which you’d like to generate methods by marking them up with tags.

A new slice type is generated for the types you specify. We call it the plural type. For example, if you gen an existing type Thing, a new plural type will be created:

type Things []Thing

…and you’ll use this wherever you would otherwise use a slice.

myThings := Things{...}
otherThings := myThings.Where(func).Sort(func)

Quick start

Of course, start by installing Go, setting up paths, etc. Then:

go get github.com/clipperhouse/gen

Create a new Go project, and cd into it. Create a main.go file and define a type in it.

Now, mark it up with a simple +gen tag in an adjacent comment like so:

// +gen
type MyType struct {
    ...
}

And at the command line, simply type

gen

You should see a new file, named mytype_gen.go. Have a look around. Try out that plural type.

Usage

If you’d like to generate methods that take pointers instead of value types, add the * directive.

// +gen *
type MyType struct {
    ...
}

To select only a subset of methods, use:

// +gen * methods:"Any,Where,Count"
type MyType struct {
    ...
}

(You’ll recognize this as the syntax of struct tags. Avoid spaces within tags.)

To include projection methods, specify projected types:

// +gen * projections:"int,OtherType"
type MyType struct {
    ...
}

Methods

Signatures use the example type *Thing.

Where

Returns a new slice (plural type) whose elements return true for passed func. Comparable to Linq’s Where and JavaScript’s filter.

func (rcv Things) Where(fn func(*Thing) bool) Things

Example:

shiny := func(p *Product) bool {
    return p.Manufacturer == "Apple"
}
wishlist := products.Where(shiny)

Count

Returns an int representing the number of elements which return true for passed func. Comparable to Linq’s Count.

func (rcv Things) Count(fn func(*Thing) bool) int

Example:

countDracula := monsters.Count(func(m *Monster) bool {
    return m.HasFangs()
})

Any

Returns true if one or more elements returns true for passed func. Comparable to Linq’s Any or underscore’s some.

func (rcv Things) Any(fn func(*Thing) bool) bool

Example:

bueller := func(s *Student) bool {
    return s.IsTruant
}
willBeHijinks := students.Any(bueller)

All

Returns true if every element returns true for passed func. Comparable to Linq’s All or underscore’s every.

func (rcv Things) All(fn func(*Thing) bool) bool

Example:

mustPass := func(t *Thing) bool {
    return !t.IsEternal
}
cliché := things.All(mustPass)

First

Returns first element which returns true for passed func. Comparable to Linq’s First or underscore’s find.

func (rcv Things) First(fn func(*Thing) bool) (*Thing, error)

Example:

come := func(c *Customer) bool {
    return c.IsHere
}
served, err := people.First(come)

Returns error if no elements satisfy the func.

Single

Returns unique element which returns true for passed func. Comparable to Linq’s Single.

func (rcv Things) Single(fn func(*Thing) bool) (*Thing, error)

Example:

id := request.Id
byId := func(t *Thing) bool {
    return t.Id == id
}
item, err := things.Single(byId)

Returns error if multiple or no elements satisfy the func.

Each

Invokes passed func on every element. Comparable to underscore’s each.

func (rcv Things) Each(fn func(*Thing))

Example:

update := func(s *Score) {
    s.Recalc()
}
scores.Each(update)

Sort

Returns a new slice (plural type) whose elements are sorted.

type SortableThing int

func (rcv SortableThings) Sort() SortableThings

Sort uses Go’s sort package by implementing the interface required to use it. Similarly, SortDesc and IsSorted(Desc).

In keeping with gen’s design, Sort returns a new sorted slice. If this is too expensive for you, you can just as easily use Go’s sort.Sort, which operates in-place.

Note: Sort will only be generated for types that can be compared greater than or less than one another (‘ordered’ in Go terminology).

SortBy

Returns a new slice (plural type) whose elements are sorted based on a func defining ‘less’. The less func takes two elements, and returns true if the first element is less than the second element.

func (rcv Things) SortBy(less func(*Thing, *Thing) bool) Things

Example:

rank := func(a, b *Player) bool {
    return a.Rank < b.Rank
}
leaderboard := player.SortBy(rank)

SortByDesc works similarly, returning the elements in reverse order.

IsSortedBy(Desc) uses a similar idiom, returning true if the elements are sorted according to the ‘less’ comparer.

Distinct

Returns a new slice (plural type) representing unique elements. Comparable to Linq’s Distinct or underscore’s uniq.

func (rcv Things) Distinct() Things

Example:

snowflakes := hipsters.Distinct()

Note: Distinct will only be generated for types that support equality.

Keep in mind that pointers and values have different notions of equality, and therefore distinctness.

DistinctBy

Returns a new slice (plural type) representing unique elements, where equality is defined by a passed func.

func (rcv Things) DistinctBy(func(*Thing, *Thing) bool) Things

Example:

hairstyle := func(a *Fashionista, b *Fashionista) bool {
    a.Hairstyle == b.Hairstyle
}
trendsetters := fashionistas.DistinctBy(hairstyle)

Min

func (rcv Things) Min() (*Thing, error)

Returns the minimum value of Things. Returns an error when invoked on an empty slice, considered an invalid operation.

Example:

type Price float64
cheapest, err := prices.Min()

In the case of multiple items being equally minimal, the first such element is returned.

Note: Min will only be generated for types that support less than/greater than (‘ordered’ in Go terminology).

MinBy

func (rcv Things) MinBy(less func(*Thing, *Thing) bool) (*Thing, error)

Returns the element containing the minimum value, when compared to other elements using a passed func defining ‘less’. Returns an error when invoked on an empty slice, considered an invalid operation.

Example:

price := func(a, b *Product) bool {
    return a.Price < b.Price
}
cheapest, err := products.MinBy(price)

In the case of multiple items being equally minimal, the first such element is returned.

Max

func (rcv Things) Max() (*Thing, error)

Returns the maximum value of slice (plural type). Returns an error when invoked on an empty slice, considered an invalid operation.

Example:

type Price float64
dearest, err := prices.Max()

In the case of multiple items being equally maximal, the first such element is returned.

Note: Max will only be generated for types that support less than/greater than (‘ordered’ in Go terminology).

MaxBy

func (rcv Things) MaxBy(less func(*Thing, *Thing) bool) (*Thing, error)

Returns the element containing the maximum value, when compared to other elements using a passed func defining ‘less’. Returns an error when invoked on an empty slice, considered an invalid operation.

Example:

area := func(a, b *House) bool {
    return a.Area() < b.Area()
}
roomiest, err := houses.MaxBy(area)

Projections

Projections allow ad-hoc mapping of one type to another.

Projections are specified by listing types in a projection tag. Here’s a marked-up type and some handy func’s, used in the examples below:

// +gen projections:"int,string"
type Thing struct { 
    Department string
    Year       int
    Sales      float64
    ...
}

getYear := func(t *Thing) int {
    return t.Year
}

getSales := func(t *Thing) float64 {
    return t.Sales
}

Select

Returns a projected slice given a func which maps Thing to {{Type}}. Comparable to Linq’s Select or underscore’s map.

Example:

names := myThings.SelectString(func(t *Thing) string {
    return t.Department
})
// => ["Widgets", "Doodads"]

GroupBy

Groups elements into a map keyed by {{Type}}. Comparable to Linq’s GroupBy or underscore’s groupBy.

Example:

report := myThings.GroupByInt(getYear)
// => { "1995": [Thing1, Thing2], "2008": [Thing3, Thing4] }

Note: Because map keys must be compared for equality, GroupBy() will only be generated for types that support equality.

Sum

Sums over all elements. Comparable to Linq’s Sum.

Example:

revenue := myThings.SumFloat64(getSales)
// => 9457846.74

Note: Sum will only be generated for numeric types.

Average

Sums over all elements and divides by len(Things). Comparable to Linq’s Average.

Example:

avg := myThings.AverageFloat64(getSales)
// => 30005.74

Note: Average will only be generated for numeric types.

Max

Selects the largest projected value. Comparable to Linq’s Max.

Example:

bigmoney := myThings.MaxFloat64(getSales)
// => 68598.99

Note: Max will only be generated for ordered types, that is, types which can be evaluated as greater or less.

Min

Selects the least projected value. Comparable to Linq’s Min.

Example:

earliest := myThings.MinInt(getYear)
// => 1995

Note: Min will only be generated for ordered types, that is, types which can be evaluated as greater or less.

Aggregate

Iterates over Things, operating on each element while maintaining ‘state’. Comparable to Linq’s Aggregate or underscore’s reduce.

Example:

var join = func(state string, value Thing) string {
    if state != "" {
        state += ", "
    }
    return state + value.Department
}
list := myThings.AggregateString(join)
// => "Sales, Marketing, Finance"

Subsetting

By default, all of the above standard methods are created when you gen a type. Similarly, all of the above projection methods are generated when you include a projections tag.

If you would prefer only to generate specific methods, you can do so with a methods tag.

// +gen methods:"Count,Where,Select,GroupBy" projections:"string,float64"
type Thing struct {
    Name  string
    Year  int  
    Sales float64
}

Negative subsetting

You can also gen every method except the ones specified by prepending the tag contents with -

// +gen methods:"-Count,Where,Select,GroupBy" projections:"string,float64"

Negative subsetting is unsupported on the projections tag.

Containers

Containers are type-specific data structures, such as lists, trees, etc. The goal is to remove the need for casting (type assertions) that are required in other implementations.

Containers are specified using the containers tag.

// +gen containers:"Set,List,Ring"
type Thing struct {
    ...
}

Set

type ThingSet map[Thing]struct{}

Implements a strongly-typed set with common operations (Union, Difference, etc). Items stored within it are unordered and unique.

The implementation is based on github.com/deckarep/golang-set, with permission. API documentation is available here. Parameters and return values that would be interface{} in the @deckarep implementation will instead use your type in the gen implementation.

List

type ThingList struct

Implements a strongly-typed, doubly-linked list, based on golang.org/pkg/container/list. API documentation is available at that link. Parameters and return values that would be interface{} in the golang implementation will instead use your type in the gen implementation.

Ring

type ThingRing struct

Implements strongly-typed operations on circular lists, based on golang.org/pkg/container/ring. API documentation is available at that link. Parameters and return values that would be interface{} in the golang implementation will instead use your type in the gen implementation.

Tips

Start with buildable code

gen parses your source and evaluates types. If your code doesn’t parse or build, gen’s ability to reason about it is nearly zero. It will return errors at the command line.

FAQ

Why?

Go doesn't (yet) offer generic types, and we are accustomed to many of their use cases. Perhaps you are similarly accustomed, or would find them useful.

Code generation, really?

Yes. We do it to ensure compile-time safety, and to minimize magic. It’s just code, and you can read it.

Codegen is not without its risks, we understand. But keep in mind that languages which do support generics are in fact doing something like code generation, so perhaps gen’s approach is not that far out.

What’s that plural type business?

gen creates a new slice type to serve as method receivers. It’s clearer and less verbose to type myThings.Where(fn) than package.Where(myThings, fn). Not to mention, the latter wouldn’t work for multiple types without the use of reflection.

gen returns a new slice for many operations

This is by design, so that there is no question about whether you are mutating a slice in place or not.

Slices are cheap. You can reduce potential allocations by operating on pointer types, as well, by using the * directive in your gen tag.

You re-implemented sort?

Yes. Go’s sort package requires the fulfillment of three interface members, two of which are usually boilerplate. If you want to sort by different criteria, you need to implement multiple ‘alias’ types.

gen’s SortBy requires a single argument defining ‘less’, and no interface implementation. You can call ad-hoc sorts simply by passing a different func.

gen’s implementation is a strongly-typed port of Go’s implementation. Performance characteristics should be similar.

We’ve also implemented a simpler Sort which requires no ‘less’ func for certain types.

Could some of this be achieved using reflection or interfaces + type assertions?

Perhaps! It’s early days and the focus is on the API and compile-time safety. We’ve no doubt that more idiomatic approaches will reveal themselves.

Not all projection methods are generated for every type.

gen only generates methods that are meaningful for a type. It won’t generate an average of strings, for example, and it won’t key a map with any type that doesn’t support equality.

What about imported types?

gen doesn’t support imported types, not least because one can’t define methods on them. You might instead create a local ‘alias’ type, something like:

type MyType otherPackage.Type

// or...

type MyType struct {
    otherPackage.Type
}

Why didn’t you implement X method?

Most likely:

Here are some design criteria.

Can I use it?

We’d be thrilled if you would test it out and offer feedback. It’s still early bits, caveat emptor for production use. The API will likely be volatile prior to 1.0.

Please let us know what you think via GitHub issues or ping Matt on Twitter @clipperhouse.

Can I help?

Sure, the code is here.

Who is ‘we’?

Matt Sherman, mostly. You can reach him @clipperhouse on GitHub or Twitter. We have are ramping up with contributors too.

Changelog

github.com/clipperhouse/gen/blob/master/CHANGELOG.md