When dealing with a huge amount of static data’s which are requested continuously, we need to have a fast key-value cache in place. Since we already had a huge chunk of data in SQL database. Let’s see how can we convert that to CSV and then to redis.

Step – 1 | Export data from SQL database

Here the data is exported to CSV format from the SQL data store using the favourite tools. We used “sequel pro” to dump the data to CSV format.

Step -2 | Convert the raw CSV data to Redis Commands (Generate the Redis commands from CSV)

Our CSV file is in the following format. Let’s call this as input.csv

1, key1, value1, created_at, updated_at
2, key2, value2, created_at, updated_at

Convert the CSV to Redis commands. This will print “SET, key1, key2” from the “input.csv” file and remove the double quotes. Edit the command to match your CSV file. “$1” prints the first column. (Counting start from 1 and not 0)

awk -F, 'BEGIN {OFS=","} { print "SET",$2, $3}' input.csv | sed 's/\"//g'

This will print the response on the command line as follows. Make sure that the output is in the format  “SET, key1,key2“. Once you are sure that the response is in the expected format, save the file to output.txt as follows

awk -F, 'BEGIN {OFS=","} { print "SET",$2, $3}' input.csv | sed 's/\"//g' > output.txt

Note: We will be using the Redis mass insert. However, you need to convert the Redis commands to Redis protocol to do that.

Step – 3 | Convert the Redis command generated to Redis protocol. (Redis protocol generator)

Here we will be generating the redis protocol to convert the redis commands for importing to redis. We will be using the redis mass insert method for that.

Copy and save the following script to gen_redis_proto.py. Raw GitHub link

#!/usr/bin/env python -tt
# -*- coding: UTF-8 -*-
"""
Generating Redis Protocol

Generate the Redis protocol, in raw format, in order to use 'redis-cli --pipe' command to massively insert/delete.... keys in a redis server
It accepts as input a pipe with redis commands formatted as "SET key value" or "DEL key"...

Usage:

      echo "SET,mykey1,value1\nSET,mykey2,value2" > data.txt
      cat data.txt | python gen_redis_proto.py | redis-cli --pipe

"""

__author__ = "Salimane Adjao Moustapha (me@salimane.com)"
__version__ = "$Revision: 1.0 $"
__date__ = "$Date: 2013/04/30 12:57:19 $"
__copyleft__ = "Copyleft (c) 2013 Salimane Adjao Moustapha"
__license__ = "MIT"

import sys
import fileinput
from itertools import imap


def encode(value):
    "Return a bytestring representation of the value"
    if isinstance(value, bytes):
        return value
    if not isinstance(value, unicode):
        value = str(value)
    if isinstance(value, unicode):
        value = value.encode('utf-8', 'strict')
    return value


def gen_redis_proto(*cmd):
    proto = ""
    proto += "*" + str(len(cmd)) + "\r\n"
    for arg in imap(encode, cmd):
        proto += "$" + str(len(arg)) + "\r\n"
        proto += arg + "\r\n"
    return proto


if __name__ == '__main__':
    for line in fileinput.input():
        sys.stdout.write(gen_redis_proto(*line.rstrip().split(',')))

Now run the following command to generate the redis protocol command and save to redis using the mass insert method.

cat output.txt | python gen_redis_proto.py | redis-cli --pipe

The CSV will be now cached into the redis memory. You can save the data in the memory to a .rdb file using “BGSAVE” command

The post How to convert and import CSV to Redis appeared first on DataYuge.

Since the launch of our Price Comparison API, the developers were always looking for building the deals and coupon codes engine. We heard you and we have introduced the Coupon Code API. This API gives a list of latest coupon codes and deals for 2000+ e-commerce + online websites. All the coupons are tested by the team and are only listed. So let’s see how to use the coupon codes API.

Step 1 – Obtain the Coupon Code API key

The first step is to obtain an API for accessing the service. You can skip this step if you already have an API key. Note that the Price Comparison API, the Coupon Codes and Deals API share a single API key which can be obtained from the PriceYuge portal.

Step 2 – Access the Coupon API endpoint

It is recommended to read the documentation well before continuing. Once you have an idea of how the platform works, you can implement it easily. The deals and coupon API have different endpoints. Let’s access the deals API endpoint first.

https://price-api.datayuge.com/api/v1/offers/list/coupons?api_key=YOUR_API_KEY

Make sure to replace the “YOUR_API_KEY” with your Coupons and Deals API Key. The above request will return a JSON response with the latest coupon codes available. The API can also pass certain parameters for refining the results to a particular category or a store. Now let’s parse the response from JSON and utilise it.

Part 2 of this tutorial will be available soon. Let’s see in the next part.

The post How to build a coupons & deals website using Coupon Code API appeared first on DataYuge.

The recharge plans API is something very crucial if your website or the app provides an online recharge service. The API is normally called browse plan API or it is also known as mobile operator plan offer finder API. DataYuge provides a daily updated recharge plan API which all the offers are daily updated, unlike the other providers. Using the free API, the developer can implement the recharge plan API on the website. This recharge plans API is totally a free solution without any limitation. However, you can see the premium plan if you wanted a completely white labelled browse plan API solution with technical assistance.

Step 1 – Obtain the recharge plans API key

DataYuge’s browse plan API works by sending a request to the secure server using an API key parameter in the URL. The developer has to obtain a free API key or a premium key by contacting DataYuge team.

Step 2

After obtaining the API key, the developer can start integrating the API into the web/mobile apps. The API looks something like this.
https://api.datayuge.com/v7/rechargeplans/?apikey=4dTA2DtUAlU5GL5XI084YWflgM&operator_id=Airtel&circle_id=Delhi

Please note that the API key used in the above browse plan API is a dummy key. You have to obtain a valid key to use them.

Step 3

This is the sample code for accessing the mobile plans API on your website. The sample code is written using basic HTML and PHP. You are welcome to modify the code as you need and implement on your website. Commenting is done for basic code readability. You can ask in comments if something you didn’t understand.

First, create a “plans.css” using the following code, which will the be CSS for the HTML page.

Now create another file and name it as “plans.php” with the following code.

This will look something like

Now you have implemented the plans API. Similarly DataYuge providers API for DTH recharge Plans and Data card recharge plans. The more details about this browse plans API can be found here. Also please check out the MNP lookup API where the operator and circle of MNP (ported numbers) can be found.

The post How to implement Free Recharge Plans API in your apps? appeared first on DataYuge.

The mobile number system in India is quite large with 8+ major providers on the market. The number allocation for each telecom carrier is different. Mainly the Indian mobile numbers start with ‘9’ or ‘8’ or ‘7’. It is a very difficult task to find the operator and circle of the mobile number. With the entry of new providers like JIO, the need for Jio number lookup API also increased. This is because of the new number allocation for these carriers. We are providing a free mobile operator finder API.

Please note: This is completely free API without any restrictions. Click here for MNP Lookup API, where you can find the operator and circle of ported numbers and even JIO numbers.

Step 1

• Get the API from here.

https://api.datayuge.com/v1/lookup/{number}

• The number parameter can be the first 5 digits of an Indian Mobile number or a valid 10 digit Indian Mobile Number

https://api.datayuge.com/v1/lookup/80000
or
https://api.datayuge.com/v1/lookup/8000012345

• The database is automatically updated, so you don’t have to worry about the accuracy of the database. As new series allocates, it’s automatically included in the API.

Step 2

The API can be easily consumed in any programming language. Hereby showing a small script on how to consume the MNP check API in PHP.

Please note that the same can be implemented in any programming language. Send an HTTP request, then decode the response and use the data. There is a rate-limiting enabled for the free API to avoid misuse of the API. The current rate-limiting is 180 5 requests per minute.

Now you can use this API to get the current operator and circle of mobile number. However, this only shows the default operator and circle. For showing the MNP Details, that is the ported operator and circle of mobile number, please try the DataYuge’s MNP API

The post How to find operator and circle of mobile number? Mobile Operator Finder API appeared first on DataYuge.