I build a lot of small web servers in Go that often need to embed static assets, usually HTML/CSS/JavaScript files. Luckily since Go 1.16 we've had the embed package as a simple and powerful tool for embedding whatever data we need into our resulting Go binaries. This functionality combined with Go's static-first approach to compilation means its trivial to build binaries that Just Work (tm) wherever you need to run them.
Lets take the most common use case I have which is simply taking some sort of bundled JavaScript project and embedding it (static assets and all) for use within a HTTP web server. Generally modern web projects using something like vite live in a sub-directory like frontend/
within my repository, keeping all the node/etc mess relatively contained. Within this directory I'll usually throw a single Go file (making it a separate package for debug/clarity purposes) like so:
package frontend
import (
"embed"
)
//go:embed dist/*
var FS embed.FS
This embeds the entirety of the built assets within dist/
into the binary as an embed.FS which implements fs.FS for wider use. Now within an HTTP server or router its trivial to mount and use this filesystem to serve assets:
// remove the dist/ prefix from the filesystem (since that comes with our embed directive)
dist, _ := fs.Sub(frontend.FS, "dist")
// serve files such that dist/example.txt would be available at /assets/example.txt
router.Mount("/assets", http.FileServer(http.FS(dist)))
//go:embed poodle.dll
var dll []byte
func GetSharedLibrary() (string, []byte) {
return "poodle.dll", dll
}
poodleName, poodleData := poodle.GetSharedLibrary()
var poodlePath string = filepath.Join(dataPath, poodleName)
if _, err := os.Stat(poodlePath); os.IsNotExist(err) {
err = ioutil.WriteFile(poodlePath, poodleData, os.ModePerm)
if err != nil {
return nil, err
}
}
Static assets are probably my prime example of embed
's usefulness, but it's actually pretty far reaching even in my own projects. A great example of embed
lifting its weight is in this code which embeds a DLL that can be extracted and loaded by the binary itself. While there is some wasted space here as the DLL eventually ends up residing on disk, it makes installation and use of the program trivial. Using embed means I was able to share the resulting .exe
across PCs without problems or users needing to install/unzip various dependencies or shared libraries.
Shandi actually makes pretty extensive use of embed
so its probably a great starting point for simple real-world examples.
go generate
//go:generate bin/gen-proto rust-server/protos/
//go:embed descriptors.bin
var descriptors []byte
var data descriptorpb.FileDescriptorSet
if err := proto.Unmarshal(descriptors, &data); err != nil {
return nil, err
}
messageTypes := make(map[string]protoreflect.MessageType)
methods := make(map[string]*descriptorpb.MethodDescriptorProto)
for _, file := range data.File {
reflectFile, err := protodesc.NewFile(file, protoregistry.GlobalFiles)
if err != nil {
return nil, err
}
protoregistry.GlobalFiles.RegisterFile(reflectFile)
for _, service := range file.Service {
for _, method := range service.Method {
methods[fmt.Sprintf("%s.%s.%s", *file.Package, *service.Name, *method.Name)] = method
}
}
for i := 0; i < reflectFile.Messages().Len(); i++ {
message := reflectFile.Messages().Get(i)
messageTypes[string(message.FullName())] = dynamicpb.NewMessageType(message)
}
}
Another great example of embed
is this code which uses go generate
to produce a descriptor file which is then embedded in the resulting binary. The binary data is available as a normal array of bytes making use trivial.
While I do agree counting bytes is useful in some scenarios, I've never needed to worry about Go binary size. Even shandi which has easily been my most heavy project from a binary-bloat perspective clocked in somewhere around 75Mb. Perhaps 10 years ago this was enough to raise an eyebrow, but today not so much. I'd still be mindful of what your embedding and try to trim things down, but depending on your use case it probably won't matter much.