I know I’ve been pretty bad lately about posting. I’m trying to be careful not to be upset that I haven’t posted in awhile and rather enjoy this–after all, this is intended to be a fun learning experience for me, and not a job.
In any case, for years I’ve been lugging around a 1TB external hard drive containing all of my music, movies and data. That’s so 2004. AWS pricing is continually falling, and it seems like that’d be a smarter choice for storing my data. The downside is that I do like to use iTunes to manage my shows, but I imagine I’ll get over that.
Upload my movies and TV shows to a personal S3 bucket, and create a web-based front-end for consuming them.
I’ve done a quick demo in Python 3, but given that I’m enjoying using Go, I’m going to do this blog post using Go.
Setting up the S3 bucket was simple for me; I already have a personal AWS account that I use to pay for the tiny EC2 instance I have hosting some domains. I did some quick math, and it seemed like 1TB of data for personal use would cost me on the order of tens of pennies per month, assuming I watched a couple hours of shows/movies per day (which I don’t).
Once the bucket was set up, I simply used the command-line tool to upload my data:
aws s3 sync /path/to/media/root s3://my-bucket-nameand then wait for quite awhile. That was pretty simple.
Well, it seems like a good start would be to list the buckets I have available. This will show me that I have the rough framework in place for working with the pieces. Following the AWS Getting Started Guide, this is a pretty simple thing to do:
package main
import (
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/aws"
"log"
"github.com/aws/aws-sdk-go/service/s3"
)
func main() {
log.Printf("Listing the buckets")
sess, err := session.NewSession()
if err != nil {
log.Fatalf("Unable to create an AWS session: %v", err)
}
// S3 service client
svc := s3.New(sess)
// List the buckets
result, err := svc.ListBuckets(nil)
if err != nil {
log.Fatalf("Unable to list the buckets: %v", err)
}
for _, b := range result.Buckets {
log.Printf("Bucket Name: %v", aws.StringValue(b.Name))
}
}
and we run it…
Right off the bat, credentials issues. Yay for test-driven development – clearly I need some AWS credentials set up. I’ve been using the GoLand IDE by JetBrains, so I simply change my run configuration to supply some environment variables (AWS_SECRET_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_REGION). Then I run the program again and I see some more useful output:
2018/08/23 22:06:26 Listing the buckets
2018/08/23 22:06:26 Bucket Name: jfisher-media-library
Ok, now we can add a bit more logic to start looking inside of the bucket for the various objects stored there.
package main
import (
"github.com/aws/aws-sdk-go/aws/session"
"log"
"github.com/aws/aws-sdk-go/service/s3"
"github.com/aws/aws-sdk-go/aws"
)
const (
MEDIA_BUCKET_NAME = "jfisher-media-library"
)
func main() {
log.Printf("Listing the buckets")
sess, err := session.NewSession()
if err != nil {
log.Fatalf("Unable to create an AWS session: %v", err)
}
// S3 service client
svc := s3.New(sess)
// List objects in the bucket
resp, err := svc.ListObjects(&s3.ListObjectsInput{Bucket: aws.String(MEDIA_BUCKET_NAME)})
if err != nil {
log.Fatalf("Unable to list the objects in the bucket: %v", err)
}
for _, item := range resp.Contents {
log.Printf("Name: %v", *item.Key)
}
}
Wow, ok, now we’re getting somewhere.
Let’s set aside the back-end and start focusing on the front-end of this for a minute. We want to create a web service
that listens for incoming requests. Fortunately, the net/http has a way to do this nicely! A very simple web server
looks like this:
package main
import (
"log"
"net/http"
)
func handler(w http.ResponseWriter, r *http.Request) {
log.Printf("Received request: %v", r)
}
func main() {
http.HandleFunc("/", handler)
log.Fatal(http.ListenAndServe(":8080", nil))
}
While this is useful, we’re going to want something that has a bit more power to it. What I’m after is something that allows better parsing of URL paths, something that will allow for named parameters ideally.
Gorilla Mux is a library that enhaces routing and dispatching of URL requests – just what we’re after.
Here is the code I’ll be building upon to search for episodes:
package main
import (
"log"
"net/http"
"github.com/gorilla/mux"
)
func getEpisode(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
show := vars["Show"]
season := vars["Season"]
episode := vars["Episode"]
log.Printf("Getting episode %v of %v, %v", episode, show, season)
}
func getEpisodes(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
show := vars["Show"]
season := vars["Season"]
log.Printf("Getting a list of episodes for season %v of %v", season, show)
}
func getSeasons(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
show := vars["Show"]
log.Printf("Getting a list of seasons available for %v", show)
}
func getShows(w http.ResponseWriter, r *http.Request) {
log.Printf("Getting a list of shows available")
}
func main() {
r := mux.NewRouter()
r.HandleFunc("/", getShows)
r.HandleFunc("/{Show}", getSeasons)
r.HandleFunc("/{Show}/{Season}", getEpisodes)
r.HandleFunc("/{Show}/{Season}/{Episode}", getEpisode)
log.Fatal(http.ListenAndServe(":8080", r))
}
Running this, and using a browser at http://localhost:8080/The Simpsons/Season 1 for instance, the console
will show “Getting a list of episodes for season Season 1 of The Simpsons”. Progress!
Ok, we have the basic pieces in place. We can query AWS to list the objects in a bucket, which is where the media files list. We can also listen for HTTP requests from browsers, parsing the URLs to extract the information we’ll use to look up the media.
Next, we’ll hook up the logic that lists the S3 objects in a given path. Then we’ll have to set up the front-end to serve up HTML that contains useful information.