Articles from July 2019

Python to Cosmos

Another step in my Python journey. This time I want to explore what it takes to work with CosmosDB in Python. I document the progress as I go and then update the overview.

In the end I have learned these:

  1. Python Package Management (PIP) – to consume packages from the community
  2. Install Python 3 and get it worked on Mac alongside with preinstalled Python 2.7
  3. Azure Cosmos Python SDK
  4. More confident in writing Python code

Things look simple and easy in abstract, in thinking. But the devil is in the detail, in the implementation level.

Python Package Management – PIP

Just like NuGet in .NET ecosystem, PIP allows developers to consume packages created by the community. According to the documentation, when Python 2.7 or 3.x is installed, PIP is installed as well.

However, when I run on Mac, it said the PIP is not a valid command. The solution is described in SO. In short, run this command

sudo -H python -m ensurepip

After that, run this command to see what you can do with PIP

pip

Most of the time, the install command is used.

pip install {package_name}

Cosmos

Cosmos has a Python SDK which is available via azure_cosmos package. There is also a step by step instruction. If one wishes to see the sample code, MS provides a sample code here

Installing the package is not going well. If this command does not work

sudo -H pip azure-cosmos

Then try this one. It works well for me

sudo -H pip azure-cosmos --upgrade --ignore-installed six

Python Versions

Mac OS has preinstalled Python version 2.7.10, so the pip is attached to the version. I want to have the latest version which is 3.7.4 at the moment. I definitely do not want to mess up the current version. Mac OS has many built in functions with Python, well at least as my guess.

brew install python

Will install the latest Python version available with the new command python3 So does the pip3. Let install azure-cosmos again for Python 3

sudo -H pip3 azure-cosmos --upgrade --ignore-installed six

Get Hands Dirty

Most of the code is out there with a few search. My objective is just to get the flow and how code is organized in Python.
The first step is to create Azure CosmosDB account first. By creating a new resource group, I can clean it everything when not in used.

Import Cosmos Package

To consume a library, it needs to be imported into the file, the same as using in C#

# Import cosmos client and alias
import azure.cosmos.cosmos_client as cosmos_client

# To ensure it is valid
print(cosmos_client)

Using alias allows us to write readable code. Instead of writing the full path azure.cosmos.cosmos_client, just write cosmos_client or whatever alias we please. Executing the above code will display the cosmos_client module

<module 'azure.cosmos.cosmos_client' from '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/azure/cosmos/cosmos_client.py'>

So far so good!

Config to Cosmos

To connect and work with Cosmos, we need a couple of information. I usually prefer to wrap them in a single config object. In Python, a dictionary is perfect for this.

# Config for cosmos database. This is a dictionary in python
'''
Working with CosmosDB requires
    ENDPOINT: The URI identified the cosmos server.
    MASTER_KEY: Authorization key, just like username/password in SQL.
    DATABASE: The name of the database.
    COLLECTION: Or CONTAINER name.
Operations are per collection in Cosmos.
'''
config = {
    'ENDPOINT':'',
    'MASTER_KEY':'',
    'DATABASE':'',
    'COLLECTION':''
}

ENDPOINT and MASTER_KEY are from Azure CosmosDB account, you should be able to find them under the Keys section.
DATABASE and COLLECTION are from users, well from me actually. I am playing around with Cosmos using Python so I want to create them dynamically. So let write some lines to ask for that

# Prompt users for database name and collection
config['DATABASE'] = input("Database: ")
config['COLLECTION'] = input("Collection: ")

Very straightforward! Ask users for input with a prompt. OK! Ready to create my very first database

# Import the library and create an alias. Look the same as namespace in C#
import azure.cosmos.cosmos_client as cosmos_client

print(cosmos_client)
# Config for cosmos database. This is a dictionary in python
'''
Working with CosmosDB requires
    ENDPOINT: The URI identified the cosmos server.
    MASTER_KEY: Authorization key, just like username/password in SQL.
    DATABASE: The name of the database.
    COLLECTION: Or CONTAINER name.
Operations are per collection in Cosmos.
'''
config = {
    'ENDPOINT':'Fill in Azure CosmosDB account endpoint',
    'MASTER_KEY':'Fill in the primary key',
    'DATABASE':'',
    'COLLECTION':''
}
# Prompt users for database name and collection
config['DATABASE'] = input("Database: ")
config['COLLECTION'] = input("Collection: ")

print(config)

# 1. Initialize the client that can talk to the server
client = cosmos_client.CosmosClient(
    url_connection=config['ENDPOINT'], 
    auth={
        'masterKey':config['MASTER_KEY']
        })

# 2. Create a database
db = client.CreateDatabase({'id':config['DATABASE']})

print(db)
  1. Initialize a client instance
  2. Ask it to create a database

So how does it look?

<module 'azure.cosmos.cosmos_client' from '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/azure/cosmos/cosmos_client.py'>
Database: Python-102
Collection: C102
{
    'ENDPOINT': 'Secret value', 
    'MASTER_KEY': 'Secret value', 
    'DATABASE': 'Python-102', 
    'COLLECTION': 'C102'
}
{
    'id': 'Python-102', 
    '_rid': 'CzhnAA==', 
    '_self': 'dbs/CzhnAA==/', 
    '_etag': '"00000900-0000-1800-0000-5d3be5650000"', 
    '_colls': 'colls/', 
    '_users': 'users/', 
    '_ts': 1564206437
}

The database Python-102 is created. The returned value is a dictionary containing essential information of the database. One important piece is the _self key, which is used to identify the database when creating a collection. Here it is, create a collection

# 3. Create a collection/container
collection = client.CreateContainer(
        db['_self'],
        {'id':config['COLLECTION']})

print(collection)

The cosmos_client exposes many APIs to work with CosmosDB. It is easy to create a document, query documents, and other advanced stuff if I wish to.

Say Something

The task sounds easy in your my head. The devil appears when I get my hands dirty in code. Thought, it is not a tough challenge. When I started to write both code and blog post at the same time, I gain triple outcome.

I started to love Python. It is a pleasure to write code in Python. I have been using Visual Studio and Windows for my career. I started to play around with Mac recently. Python helps me comfortable with Mac. This post is about Python and Cosmos. But I have to write my feeling after all.

What’s next? Write some utilities for myself. The goal is still to explore more about Python.

Welcome to Python

Have been working with C# for a decade, it is time to learn a new language. Some options popping over my head, of course they must be different than C# – Java is excluded immediately, are Python, Ruby on Rails, Closure, … kind of scripting languages. I know some about JavaScript. So I definitely do not want to dig deeper into it.

I remember a statistic from StackOverflow 2018, Python has a good position. I also heard somewhere that Python is awesome. So here I come Python.

If one wishes to know what Python is, ask Google. It gives you all the awesome resources ranging from introduction to intermediate to advanced. The community is strong as well.

The post is my journey, my thoughts when I started to learn Python. It also serves as my documentation. If someone ask me how to get started to Python, I can give them the link. Well, at least that what I thought.

So let’s get started!

Environment

Each developer has their own favorite environments. I come from the .NET world so Visual Studio Code is my tool. Of course, I can use Visual Studio. But seems Visual Studio Code is a better choice.

  1. Visual Studio Code + Python Extension
  2. Install Python on Windows. Download and follow the instruction here at python org

Installing both Visual Studio Code and Python are fast. Everything is ready in a matter of minutes.

Quick Overview

Some notes about python:

  1. Python file extension is py Ex: hello-python.py.
  2. Can take a single Python file and run with python command line.
  3. Interactive shell is powerful. From the console/terminal/PowerShell, type the command python and the Python shell is ready.
  4. Dynamic type. The type is known at the execution time.
  5. Write "Hello World" program in a single line of code: print("Hello World") directly from the shell.

I can write functional code or object oriented code.

Everything in Python is an object. There are 2 functions that I find very useful – type() and id().

name = "Thai Anh Duc"

duplicatedname = "Thai Anh Duc"

# Want to know the type?
type(name)

# Want to know where it is store in the memory?
id(name)

# Are they equal? Are they the same?
name == duplicatedname

So name equals duplicatedname. But they are not the same. By using the id() function, we know that are store in different locations.

Materials

I learned from:

  1. Pluralsight Courses: It is always my first and default option when learning technologies.
  2. Tech Beamer: Very detail with examples, explanations.

Conventions

Python uses indentation for code block instead of square brackets ({}) in C#. There is a coding guidelines, standard PEP8. At the beginning, I do not worry too much about them. I just focus on some simple building blocks. And fortunately, the Visual Studio Code has helped me checked and suggested correction.

This block of code is enough for me to remember what are important

if(1 > 0):
    print("Of course, it is.")
for index in [1, 2, 3, 4]:
    print("hello {0}. This is a loop, display by string format".format(index))
    print("Same code block")

def i_am_a_function(parameters):
    # Comment what the function does
    print(parameters)

Data Types

Date types are crucial to code, even with a dynamic type language. As a developer, you have to know what kind of data you are dealing with. Are we talking about a string, an integer, a big decimal, or a date, …

Refer to Tech Beamer for detail of each type. A special note is at the number types. There are 3 types:

  1. int – for integer value. There is no limit as far as the computer can hold it. Unlike C#, there are limit on each number type, such as Int16, Int32, Int64.
  2. float – for floating value. There is a limit of 15 decimal digits.
  3. complex – for other cases. For example, the operator 7/3, instead of returning the value 2.333333, it is represented as (2,1) – this is the complex type.

Because type is dynamic, sometimes it is a good idea to check the type first. Python has isinstance function

name = "Thai Anh Duc"
# It is a string
isinstance(name, str)

age = 36
# It is an int
isinstance(age, int)

bank_amount = 1.1
# It is a float
isinstance(bank_amount, float)

Function and Class

One might not need a class but function.

def append_lower(name):
    # Lower case the name and then append " python" to the end
    return name.lower() + " python"

class Python101():
    # User-defined classes should be camel case
    static_constant_field = "Static field, access at the class level"
    def __init__(self, comment)
        # Constructor
        # :param self: this in C# language
        # :param comment: constructor parameter. Which tells Python that it requires a parameter
        self.comment = comment # Declare an instance field and assign value

    def say_hi(self)
        print(self.comment)

print(Python101.static_constant_field)

p101 = Python101("You are awesome")
p101.say_hi()

The code speaks for itself.

Summary

It is quite easy for a C# developer to get started with Python. With the above building blocks, I am ready to write some toy programs to play around with Python. There are many that I want to see how Python does. The initial list are:

  1. IO
  2. Network
  3. Threading

The list goes on as I learn. I am ready to go next.