When we left off last, I had set up a Go server that would listen for requests and parse out the important things we’re interested in (show, season, episode). Today I’m going to try to set up some more of the browsing code.
My S3 bucket is structured as follows:
<Show>/
<Season>/
<Episode>.mp4
This will allow me to set up a very simple file browser. One thing to be aware of is that S3 buckets aren’t traditional folders. While they support path-like object names and the look of a folder, they’re not really as simple as that. To browse, we have to deal with that.
What we’re looking for is basically a listing of the top-level objects in the S3 bucket. If we treat the object name
as if it were an actual path, we’re looking for something like ls /. I’m going to start with the code from the
previous post wherein I listed objects in a bucket:
sess, err := session.NewSession()
if err != nil {
log.Fatalf("Unable to create an AWS session: %v", err)
}
// S3 service client
svc := s3.New(sess)
resp, err := svc.ListObjects(&s3.ListObjectsInput{Bucket: aws.String(MEDIA_BUCKET_NAME)})
if err != nil {
log.Fatalf("Unable to list the objects in the bucket: %v", err)
}
for _, item := range resp.Contents {
log.Printf("Name: %v", *item.Key)
}
However, when I run this, I’ll get a list of everything in the bucket! This would be very inefficient and expensive, given that it’d require me to receive this list from S3. However, we can be a little clever here. We’re going to use the “Delimiter” field in the request to limit the request to summarize all of the paths after a common prefix (see the AWS S3 documentation):
resp, err := svc.ListObjects(&s3.ListObjectsInput{
Bucket: aws.String(MEDIA_BUCKET_NAME),
Delimiter: aws.String("/"),
})
if err != nil {
log.Fatalf("Unable to list the objects in the bucket: %v", err)
}
for _, prefix := range resp.CommonPrefixes {
log.Printf("%v", *prefix.Prefix)
}
Great: now we get a list of all of the common prefixes! Note: this does not handle the case where we have so many top-level objects that we have to use a paginated approach. I’m choosing to accept this for now.
Ok, so this is only half of the battle: we can retrieve the list of shows, but we’re still not serving anything to the browser. Let’s change that. What we want for this is to serve up a template. Enter “html/template”.
Instead of trying to explain templating, I’ll simply link you off to the documentation. The concept is pretty simple: we set up an HTML template, and later we pass in a data structure and use it to populate the HTML template with the final information we want to serve up to the user.
The HTML for this is pretty simple, and I set it up as a package-level constant:
const (
showsTemplateHTML = `
<html>
<head>
<title>List of Shows</title>
<style>
body {
font-family: sans-serif;
}
</style>
</head>
<body>
<table border=0>
{{ range . }}
<tr>
<td>
<a href="{{.}}">
{{ . }}
</a>
</td>
</tr>
{{ end }}
</table>
</body>
`
)
var (
showsTemplate *template.Template
)
The dots are pretty crappy and make for a difficult-to-understand template. Truth be told, I don’t fully understand them. Basically it seems
like shorthand for “whatever this is supposed to be”. In this case, the intention is for a string slice to be handed in, and we will iterate
over it, using each entry for a link and text in the table.
Note that I also set up the template variable to hold a reference to the compiled template. Since the template isn’t going to change at runtime,
I compile the template beforehand, in the main() function:
showsTemplate = template.Must(template.New("showList").Parse(showsTemplateHTML))
Later, when we’re serving up a response to a web browser request, we can simply do the following:
shows := make([]string, 0, len(resp.CommonPrefixes))
for _, prefix := range resp.CommonPrefixes {
shows = append(shows, *prefix.Prefix)
}
showsTemplate.Execute(w, shows)
This will execute the HTML template we precompiled with a string slice containing the names of the shows in the S3 bucket.
I ran this code and indeed I did see a simple HTML page listing the top-level “directories” in the S3 bucket!
It’s time to wrap up for now – we have family visiting. What I’ve done so far here is to find a way to parse just the top-level of the S3 “directory”, and use the “html/template” Go package to render an HTML response to the browser. Next will be extending this pattern to the next two levels: listing show seasons and listing show episodes.
For reference, here is my final code:
package main
import (
"html/template"
"log"
"net/http"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
"github.com/aws/aws-sdk-go/aws"
"github.com/gorilla/mux"
)
const (
MEDIA_BUCKET_NAME = "jfisher-media-library"
)
var (
showsTemplate *template.Template
)
const (
showsTemplateHTML = `
<html>
<head>
<title>List of Shows</title>
<style>
body {
font-family: sans-serif;
}
</style>
</head>
<body>
<table border=0>
{{ range . }}
<tr>
<td>
<a href="{{.}}">
{{ . }}
</a>
</td>
</tr>
{{ end }}
</table>
</body>
`
)
func getEpisode(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
show := vars["Show"]
season := vars["Season"]
episode := vars["Episode"]
log.Printf("Getting episode %v of %v, %v", episode, show, season)
}
func getEpisodes(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
show := vars["Show"]
season := vars["Season"]
log.Printf("Getting a list of episodes for season %v of %v", season, show)
}
func getSeasons(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
show := vars["Show"]
log.Printf("Getting a list of seasons available for %v", show)
}
func getShows(w http.ResponseWriter, r *http.Request) {
log.Printf("Getting a list of shows available")
sess, err := session.NewSession()
if err != nil {
log.Fatalf("Unable to create an AWS session: %v", err)
}
// S3 service client
svc := s3.New(sess)
resp, err := svc.ListObjects(&s3.ListObjectsInput{
Bucket: aws.String(MEDIA_BUCKET_NAME),
Delimiter: aws.String("/"),
})
if err != nil {
log.Fatalf("Unable to list the objects in the bucket: %v", err)
}
shows := make([]string, 0, len(resp.CommonPrefixes))
for _, prefix := range resp.CommonPrefixes {
shows = append(shows, *prefix.Prefix)
}
showsTemplate.Execute(w, shows)
}
func listBuckets() {
sess, err := session.NewSession()
if err != nil {
log.Fatalf("Unable to create an AWS session: %v", err)
}
// S3 service client
svc := s3.New(sess)
// List objects in the bucket
resp, err := svc.ListObjects(&s3.ListObjectsInput{Bucket: aws.String(MEDIA_BUCKET_NAME)})
if err != nil {
log.Fatalf("Unable to list the objects in the bucket: %v", err)
}
for _, item := range resp.Contents {
log.Printf("Name: %v", *item.Key)
}
}
func main() {
// Pre-compile the HTML templates
showsTemplate = template.Must(template.New("showList").Parse(showsTemplateHTML))
r := mux.NewRouter()
r.HandleFunc("/", getShows)
r.HandleFunc("/{Show}", getSeasons)
r.HandleFunc("/{Show}/{Season}", getEpisodes)
r.HandleFunc("/{Show}/{Season}/{Episode}", getEpisode)
log.Fatal(http.ListenAndServe(":8080", r))
}