I run a Whatsapp newsletter where I share useful resources and articles about products and startups. But the problem with Whatsapp Newsletter is, new subscribers can’t access the old content.

So, I decided to create a simple project where you can access all the articles/resources which I have shared with the group, segregated into different sections.

Building a curation website like this is super easy.

By following this tutorial you will create your own curation website and will learn about creating different endpoints in the backend to access data on HarperDB.

For the backend, I was planning on using an Express server and deploying it to a Digital Ocean instance.

Luckily, we don’t need to spend extra cash on running another instance somewhere else.

Because HarperDB recently announced Custom Functions which lets you write endpoints to access and process data. It runs a Fastify server behind the scenes and can be deployed in the same instance as your DB. So I will be using this as the backend for our website.

Things To Do

Here are a few things we need to set up to create our own curation web app:

• Create a HarperDB Cloud instance
• Create a schema and table for our data
• Create a custom function project
• Create different endpoints

Creating HarperDB Cloud Instance 

First Sign up here studio.harperdb.io

Then click on create new HarperDB Cloud Instance:

Here we will be choosing cloud instances:

There are different Instance plans here, I will choose the free option for now:

Congratulations! You have successfully completed the first step of creating a cloud instance. You are now the owner of your very own HarperDB Cloud Instance!

Creating Schema and Table 

Let’s now create a table -> create a schema and -> add data.

Open your Cloud Instance and create a schema for your table. I will name it “example”:

Now create a table, I will name it “resources” and for the hash attribute, I will be using “id”:

Time to add data… Click on the plus symbol:

And here you can either send in one JSON object or multiple ones in an array. I will be pasting the below JSON:

[
  {
    "url": "https://matthewmcateer.me/",
    "title": "A Blog on AI and Blockchain",
    "description": "Biologist-turned-ML-Engineer in San Francisco. I blog about machine learning, biotech, blockchain, dogs, and more.",
    "cover": "https://matthewmcateer.me/photo-large.png",
    "category": "AI"
  },
  {
    "url": "https://future.a16z.com/a-taxonomy-of-tokens-distinctions-with-a-difference/",
    "title": "Designing Internet-Native Economies: A Guide to Crypto Tokens - Future",
    "description": "Crypto protocols, social apps, online communities, and marketplaces will need to deeply understand the interplay between fungible and non-fungible tokens to create their own internet economies.",
    "cover": "https://future.a16z.com/wp-content/uploads/2021/06/V7_GenericTopic_Crypto.jpg",
    "category": "Crypto"
  },
  {
    "url": "https://www.youtube.com/watch?v=R9ch1Jvaj7A",
    "title": "How to Create a Killer Marketing Message",
    "description": "how to go to any event n leave with potential clients",
    "cover": "https://img.youtube.com/vi/R9ch1Jvaj7A/maxresdefault.jpg",
    "category": "Marketing"
  }
]

Now we have successfully added data to our table. Time to create some API endpoints which we can use in the frontend.

Creating Your First Custom Function 

Head over to the Functions Section and create a Project. I will name my project API.

This gives you a file for adding routes, a file for adding helper functions, and a static folder. I will discuss helper functions later in the tutorial, Right now head over to the example.js file in the Routes folder and you will see some template code given. Let’s remove it for now and just send “Hello world” as a response to check if our endpoint works.

'use strict';


module.exports = async (server) => {

  server.route({
    url: "https://dzone.com/",
    method: 'GET',
    handler: () => {

      return "Hello World!";
    }
  });

}

This will send Hello World as the output when we hit our endpoint. You can access this endpoint at -> {baseURL}/{projectName}

In our case it is functions-cloud-hsb.harperdbcloud.com/api

Nice, It’s working!

Now let’s create a route to fetch data by getting categories from params.

{baseurl}/api/resources/:category

Here we are using a SQL query to select all the documents in the table resources which is from schema “example” where “category” comes from our params.

server.route({
    url: '/resources/:category',
    method: 'GET',

    handler: (request) => {
      request.body= {
        operation: 'sql',
        sql: `select * from example.resources where category="${request.params.category}"`
      };
 return hdbCore.requestWithoutAuthentication(request);

    }
  });

Also, notice that HarperDB allows access to the database with no authentication with their built-in method. If you are doing a POST request, do create some prevalidation that checks auth token, etc.

To showcase this, I will create a simple pre-validator like throwing an error if a particular category is not available in our list.

For this, we can use helper functions. Let’s create one called “checkForCategory”

Now import the function and send the request through the prevalidation:

'use strict';
const checkForCategory = require('../helpers/checkForCategory');

  server.route({
    url: '/resources/:category',
    method: 'GET',
    preValidation: (request) => checkForCategory(request, logger),


    handler: (request) => {
      request.body= {
        operation: 'sql',
        sql: `select * from example.resources where category="${request.params.category}"`
      };



      return hdbCore.requestWithoutAuthentication(request);


    }
  });

Congratulations! You have successfully created a few endpoints to get data from your database. While also being hosted in the same instance. So, no need to deploy your backend somewhere else.

You can also do pagination here like:

server.route({
    url: '/resources',
    method: 'GET',
    handler: (request) => {
      const size = request.query.size|| 10;
      const page = request.query.page || 0;
      const skip = parseInt(page)*parseInt(size);



      request.body= {
        operation: 'sql',
        sql: `SELECT * FROM example.resources ORDER BY category LIMIT ${size} OFFSET ${skip}`
      };
      return hdbCore.requestWithoutAuthentication(request);
    }
  })};

Frontend 

I sketched a simple UI using excalidraw:

import './App.css';
import { useState,useEffect } from 'react';

function App() {
  const [items, setItems] = useState([]);
  const [activeIndex, setActiveIndex] = useState("Marketing");
  const [selectedTag,setSelectedTag] = useState("Marketing")
  useEffect(()=>{
    function fetchData () {

      fetch(`https://functions-cloud-hsb.harperdbcloud.com/api/resources/${selectedTag}`, {
    method: 'GET', // or 'PUT'


  })
  .then(response => response.json())
  .then(data => {
    setItems(data);
  })
    }
    fetchData()


  },[selectedTag]);



  const tags=["Marketing","Startups","Podcasts","Product","Crypto","Learn"];



function ChangeResource(tag){

  setSelectedTag(tag);
  setActiveIndex(tag);
}

  return (
    <> 
    <div className="row">
     <div className="Boxtitle">
      <h1> Curated List of Best Articles and Resources </h1>
     </div>
     </div>

     <div className="Boxcontent">

     {tags.map(tag=>(
       <div onClick={() => ChangeResource(tag)} key={tag} className={activeIndex === tag ? "sectionType active" : "sectionType"}> 
       <h3  > {tag} </h3>
       </div>
     ))}

     </div>
<div className="gridWrap"> 
 {items.map((item,key)=>{


    return(
        <div key={item.title} className="GridView">
     <a href={item.url} >
     <img   className="ImageStyle" src={item.cover} alt="" /> 
     <h3 style={{color:"white",width:"100%",maxWidth:"400px",textAlign:"center",display:"absolute"}}>  {item.title}  </h3>
     </a>
  </div> ) }
  )} 

    </div>





    </>
  );
}

export default App;

Here is the frontend code repo.

Here it is deployed on vercel-> smart-internet.vercel.app

(Fun Fact: You can deploy your frontend on HarperDB too if it is static. Read the FAQ section to know more). You can find the backend code here.

Misc 

Can we upload our own custom functions?

Yes, you can. Your local code should have routes and a proper project structure. You can start by cloning the official repo and changing the code along the way.

How to upload custom functions to HarperDB?

At the time of writing this article, the best way to upload your local code to your cloud instance without having a local instance running is by creating a .tar file using 7zip or any tool and then converting it to a base64 string by uploading your tar to this website.

Then you need to make a POST request to your cloud instance. You will find your auth header and instance URL in the “config” section of your HarperDB instance.

Here is the request body with project property as api (since our project name was api) and the payload will be that long base64 encoded tar text. The file section is dummy but necessary so just copy what I have written.

{
    "operation": "deploy_custom_function_project",
    "project": "api",
    "file": "/tmp/144275e9-f428-4765-a905-208ba20702b9.tar",
    "payload": "aGVscGVycy8AAAAAAAAAAAAAAAAAAAAAAA........"
}

Congratulations, you just deployed a cloud function from your local project.

I hope you learned something new today. If you did, do give this a Like. Let me know how you will use HarperDB to create your next project.