Golang: Prevent over writing file with md5 hash

This is an additional note for previous tutorial on how to upload/submit file from command line to a server. In the receiving server program, we overwrite the file upon receiving. There are times when we need to check first if the file that we receive from the client is the same file. If the file is same, then we do not need to save the file on server.

Now, do not confuse checking if a file exist is similar to checking if a new file is the same as the old file. Exist != Same.

To check if the new file that our server program just received is the same as the old file, we need to compute the MD5(can be sha1, sha512 or sha256 too) hash and check if any file in the directory has the same MD5 hash. If no, then proceed to overwrite. If yes, ignore the uploaded file or you can save the new file but with incremental filenames. See https://www.socketloop.com/tutorials/golang-increment-string-example

The code example that follows is a modification of a file receiving program from the command line upload file tutorial.

checkSameFile.go

 package main

 import (
 "crypto/md5"
 "fmt"
 "io"
 "io/ioutil"
 "net/http"
 "os"
 )

 func receiveHandler(w http.ResponseWriter, r *http.Request) {

 // the FormFile function takes in the POST input id file
 file, header, err := r.FormFile("fileUploadName")

 if err != nil {
 fmt.Fprintln(w, err)
 return
 }

 // sanity check on the MIME FileHeader
 fmt.Println(header.Header)

 defer file.Close()

 out, err := os.Create("/tmp/uploadedFile")
 if err != nil {
 fmt.Fprintf(w, "Unable to create the file for writing. Check your write access privilege")
 return
 }

 defer out.Close()

 // write the content from POST to the file
 _, err = io.Copy(out, file)
 if err != nil {
 fmt.Fprintln(w, err)
 }

 // now we want to check if the new uploaded file is a duplicate of our target file
 // at /tmp/keepThisFile.  If yes, abort writing to target file

 // modified from https://www.socketloop.com/tutorials/golang-find-duplicate-files-with-filepath-walk

 oldFile, err := ioutil.ReadFile("/tmp/keepThisfile")

 // target file not found
 if err != nil {
 fmt.Fprintln(w, "Old file not found, proceed to rename /tmp/uploadedFile to /tmp/keepThisFile. \n")

 // rename /tmp/uploadedFile to /tmp/keepThisFile
 os.Rename("/tmp/uploadedFile", "/tmp/keepThisFile")
 fmt.Fprintln(w, "Renamed /tmp/uploadedFile to /tmp/keepThisFile. You can proceed to digest the file for further analysis. \n")
 return

 } else {
 // target file found, get the MD5 checksum of target / old file
 oldHexDigest := md5.Sum(oldFile)
 fmt.Fprintf(w, "Old file MD5 checksum is %x \n", oldHexDigest)

 newFile, err := ioutil.ReadFile("/tmp/uploadedFile")
 if err != nil {
 fmt.Fprintln(w, err)
 }

 newHexDigest := md5.Sum(newFile)

 fmt.Fprintf(w, "New file MD5 checksum is %x \n", newHexDigest)

 if newHexDigest != oldHexDigest {

 // rename /tmp/uploadedFile to /tmp/keepThisFile
 // don't worry - /tmp/uploadedFile will be deleted in the process

 os.Rename("/tmp/uploadedFile", "/tmp/keepThisFile")
 fmt.Fprintln(w, "Renamed /tmp/uploadedFile to /tmp/keepThisFile. You can proceed to digest the file for further analysis. \n")
 fmt.Fprintf(w, "File [%s] uploaded.", header.Filename)
 return
 } else {
 fmt.Fprintf(w, "%s is a duplicate of previously uploaded file.", header.Filename)
 return
 }
 }

 }

 func page(w http.ResponseWriter, r *http.Request) {
 html := `  <html>
 <title>Upload your submission here.</title>
 <body>

 <h1>This the web way of submitting/uploading file</h1>
 <br>
 <form action="http://localhost:8888/submit" method="post" enctype="multipart/form-data">
 <label for="file">Filename:</label>
 <input type="file" name="fileUploadName" id="fileUploadName">
 <input type="submit" name="submit" value="Submit">
 </form>

 </body>
 </html>`

 w.Write([]byte(fmt.Sprintf(html)))
 }

 func main() {
 http.HandleFunc("/submit", receiveHandler)
 http.HandleFunc("/", page)
 http.ListenAndServe(":8888", nil)
 }

Build checkSameFile.go and run on the background. Point your browser to http://localhost:8888/ and upload a file to see the output message for the first time.

Next, try uploading the same file, but with a different name. If all goes well, the server detects that it is a duplicate file.

Happy coding!

References:

https://www.socketloop.com/tutorials/golang-find-duplicate-files-with-filepath-walk

https://www.socketloop.com/tutorials/golang-increment-string-example

https://www.socketloop.com/tutorials/golang-generate-md5-checksum-of-a-file

https://socketloop.com/tutorials/golang-check-if-a-file-exist-or-not