<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Yoma's Digital Garden]]></title><description><![CDATA[Hello! Welcome to my garden, where I cultivate and share my knowledge about things I find interesting.]]></description><link>https://yomaokobiah.com</link><image><url>https://cdn.hashnode.com/res/hashnode/image/upload/v1590774445230/gNKQGoZwP.png</url><title>Yoma&apos;s Digital Garden</title><link>https://yomaokobiah.com</link></image><generator>RSS for Node</generator><lastBuildDate>Sat, 18 Apr 2026 10:49:03 GMT</lastBuildDate><atom:link href="https://yomaokobiah.com/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[How to Use the CUDO Compute CLI for Virtual Machine Lifecycle Management]]></title><description><![CDATA[CUDO Compute is a decentralised cloud computing platform that provides scalable and cost-effective computing power by utilising underutilised hardware resources from data centers, enterprises, and individual contributors. It allows users to deploy an...]]></description><link>https://yomaokobiah.com/how-to-use-the-cudo-compute-cli-for-virtual-machine-lifecycle-management</link><guid isPermaLink="true">https://yomaokobiah.com/how-to-use-the-cudo-compute-cli-for-virtual-machine-lifecycle-management</guid><category><![CDATA[Cloud Computing]]></category><category><![CDATA[bash script]]></category><dc:creator><![CDATA[Yoma Okobiah]]></dc:creator><pubDate>Wed, 19 Mar 2025 12:09:16 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1741621099214/684b693d-f6d8-436f-b6ad-d4a46d3e3c60.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>CUDO Compute is a <strong>decentralised cloud computing platform</strong> that provides scalable and cost-effective computing power by utilising underutilised hardware resources from data centers, enterprises, and individual contributors. It allows users to deploy and manage virtual machines (VMs) and workloads on a distributed network of compute nodes.</p>
<p>In this tutorial, you will learn how to integrate Cudo Compute into your workflow for scalable, cost-effective computing.</p>
<h2 id="heading-objectives">Objectives</h2>
<ul>
<li><p>Create virtual machines (VMs) with CUDO’s CLI</p>
</li>
<li><p>SSH into these VMs to run a benchmark test via the CLI</p>
</li>
<li><p>Destroy the VMs via the CLI</p>
</li>
</ul>
<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">A <a target="_self" href="https://compute.cudo.org/auth/sign-in">CUDO account</a> will be required for this tutorial.</div>
</div>

<h2 id="heading-get-your-api-key">Get your API key</h2>
<p>You will need to obtain your API keys from the <code>API Keys</code> section on your CUDO dashboard.</p>
<p><img src="https://res.cloudinary.com/ichtrojan/image/upload/v1741609640/1_hh4jho.png" alt="api" /></p>
<p>Click on the “Create an API Key” button and assign a name to your API key. Your chosen API key name must be in lowercase and can be hyphenated for readability.</p>
<p><img src="https://res.cloudinary.com/ichtrojan/image/upload/v1741609640/3_icdjzo.png" alt="name" /></p>
<p>On doing this, you’ll be given an API key. Take note of this section of the app, as you’ll need to copy your API key and use it in your CLI.</p>
<p><img src="https://res.cloudinary.com/ichtrojan/image/upload/v1741609640/4_ayeuxp.png" alt="gen" /></p>
<h2 id="heading-create-your-billing-account">Create your billing account</h2>
<p>Billing accounts pay for the use of resources in Cudo Compute. To use the cloud resources in a project, the project must be linked to an active billing account. An active billing account has a credit balance; the minimum deposit is $10.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1741658109579/8cf135f9-413e-4071-a515-c27436f01298.png" alt class="image--center mx-auto" /></p>
<h2 id="heading-setup-cudo-cli">Setup CUDO CLI</h2>
<p>CUDO Compute has a CLI tool called <code>cudoctl</code> to manage all of your resources within the platform. Follow the steps below to get started.</p>
<h3 id="heading-step-1-download-the-binary-file">Step 1: Download the binary file</h3>
<p>To access the CLI’s functionality, you must download the binary file onto your local machine. To do this, go to the <a target="_blank" href="https://www.cudocompute.com/docs/cli-tool">download page</a>.</p>
<p><img src="https://res.cloudinary.com/ichtrojan/image/upload/v1741610112/5_ujajjx.png" alt="down" /></p>
<h3 id="heading-step-2-install-the-binary-file">Step 2: Install the binary file</h3>
<p>After successfully downloading the binary file, you have to move it to the right path on your machine.</p>
<pre><code class="lang-bash">chmod +x &lt;filepath&gt; 
sudo mv &lt;filepath&gt; /usr/<span class="hljs-built_in">local</span>/bin/
sudo mv /usr/<span class="hljs-built_in">local</span>/bin/&lt;filename&gt; /usr/<span class="hljs-built_in">local</span>/bin/&lt;newname&gt;
</code></pre>
<p>Replace <code>&lt;filepath&gt;</code> with the path to the downloaded binary and <code>&lt;newname&gt;</code> with <code>cudoctl</code>.</p>
<h3 id="heading-step-3-initialise-the-installation">Step 3: Initialise the installation</h3>
<p>Run <code>cudoctl init</code> and follow the steps.</p>
<pre><code class="lang-bash">cudoctl init
   ✔ api key: my-api-key
   ✔ project: my-project
   ✔ billing account: my-billing-account
   ✔ context: default
   config file saved ~/.config/cudo/cudo.yml
</code></pre>
<p>A config file will be maintained in <code>$HOME/.config/cudo/cudo.yml</code>.</p>
<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">Access help at any level by using <code>-h</code> e.g <code>cudoctl -h</code></div>
</div>

<h3 id="heading-step-4-add-ssh-key">Step 4: Add SSH Key</h3>
<p>An <strong>SSH key</strong> is used to securely access a <strong>VM</strong> over the Internet or a private network. If you don’t have an existing SSH key, follow the instructions <a target="_blank" href="https://www.cudocompute.com/docs/tutorials/how-to-generate-ssh-keys">here</a> to generate a new one on your local machine.</p>
<p>Run the following to initialise CUDO with your SSH key:</p>
<pre><code class="lang-bash">cudoctl ssh-keys init
</code></pre>
<p>Select your desired SSH key and proceed with <code>&lt;enter/return</code></p>
<p><img src="https://res.cloudinary.com/ichtrojan/image/upload/v1741611064/6_rf4l5f.png" alt="ssh" /></p>
<h3 id="heading-executing-the-bash-script">Executing the Bash Script</h3>
<p>Here’s a <a target="_blank" href="https://gist.github.com/yomaokobiah/5da837f38fe9d1d9423ce55d34a034f9">Bash script</a> that uses only the <strong>Cudos CLI</strong> to meet the objectives of this tutorial:</p>
<p>We begin with the script shebang (<code>#!/bin/bash</code>), which specifies that the script should be executed using Bash. The <code>set -e</code> command ensures that the script exits immediately if any command fails. Next, the user is prompted to enter a project name, which is used to create a project. Since all resources in Cudo Compute must be associated with a project, a VM cannot be made without first setting up a project.</p>
<pre><code class="lang-bash"><span class="hljs-meta">#!/bin/bash</span>
<span class="hljs-built_in">set</span> -e
<span class="hljs-built_in">read</span> -p <span class="hljs-string">"Enter the project name: "</span> project_name
cudoctl projects create <span class="hljs-variable">$project_name</span>
...
</code></pre>
<p>The <code>CONFIGS</code> array stores multiple VM configurations, with each entry following a structured format: <strong>VM name, data center, machine type, OS image, number of vCPUs, memory size, and disk size</strong>. Additional configuration parameters can be found in the <a target="_blank" href="https://www.cudocompute.com/docs/cli-tool">documentation</a> if needed.</p>
<pre><code class="lang-bash">...
CONFIGS=(
  <span class="hljs-string">"benchmarkvm1 us-santaclara-1 intel-broadwell ubuntu-2204-nvidia-535-docker-v20241017 1 1 10"</span>
  <span class="hljs-string">"benchmarkvm2 us-santaclara-1 intel-broadwell ubuntu-2204-nvidia-535-docker-v20241017 2 2 15"</span>
  <span class="hljs-string">"benchmarkvm3 us-santaclara-1 intel-broadwell ubuntu-2204-nvidia-535-docker-v20241017 4 4 20"</span>
)
...
</code></pre>
<p>We create a file (<code>benchmark_results.txt</code>) to store the benchmark results and add a header row (<code>VM Name, VCPU, Memory, Disk_Size, CPU Events/sec</code>) to ensure the data is easily identifiable.</p>
<pre><code class="lang-bash">...
RESULTS_FILE=<span class="hljs-string">"benchmark_results.txt"</span>
<span class="hljs-built_in">echo</span> <span class="hljs-string">"VM Name, VCPU, Memory, Disk_Size, CPU Events/sec"</span> &gt; <span class="hljs-variable">$RESULTS_FILE</span>
...
</code></pre>
<p>We define the Bash function <code>create_vm()</code> using <strong>local</strong> variables to store the function arguments, representing the array’s VM configuration details. The function uses <code>echo</code> to display status messages, providing feedback on the VM creation process. The <code>VM_ID</code> variable captures the output of the <code>cudoctl vm create</code> command, allowing us to verify whether the VM was successfully created.</p>
<pre><code class="lang-bash">...
<span class="hljs-function"><span class="hljs-title">create_vm</span></span>() {
  <span class="hljs-built_in">local</span> vm_name=<span class="hljs-variable">$1</span>
  <span class="hljs-built_in">local</span> data_center=<span class="hljs-variable">$2</span>
  <span class="hljs-built_in">local</span> machine_type=<span class="hljs-variable">$3</span>
  <span class="hljs-built_in">local</span> image=<span class="hljs-variable">$4</span>
  <span class="hljs-built_in">local</span> vcpu=<span class="hljs-variable">$5</span>
  <span class="hljs-built_in">local</span> memory=<span class="hljs-variable">$6</span>
  <span class="hljs-built_in">local</span> disk_size=<span class="hljs-variable">$7</span>

  <span class="hljs-built_in">echo</span> <span class="hljs-string">"Creating VM: <span class="hljs-variable">$vm_name</span> (<span class="hljs-variable">$vcpu</span> CPUs, <span class="hljs-variable">$memory</span> RAM, <span class="hljs-variable">$disk_size</span> Disk) in <span class="hljs-variable">$data_center</span>..."</span>

  VM_ID=$(cudoctl vm create -project <span class="hljs-string">"<span class="hljs-variable">$project_name</span>"</span> -id <span class="hljs-string">"<span class="hljs-variable">$vm_name</span>"</span> -data-center <span class="hljs-string">"<span class="hljs-variable">$data_center</span>"</span> -machine-type <span class="hljs-string">"<span class="hljs-variable">$machine_type</span>"</span> \
          -image <span class="hljs-string">"<span class="hljs-variable">$image</span>"</span> -vcpus <span class="hljs-string">"<span class="hljs-variable">$vcpu</span>"</span> -memory <span class="hljs-string">"<span class="hljs-variable">$memory</span>"</span> -boot-disk-size <span class="hljs-string">"<span class="hljs-variable">$disk_size</span>"</span>)

  <span class="hljs-keyword">if</span> [[ <span class="hljs-variable">$VM_ID</span> -gt 0 ]]; <span class="hljs-keyword">then</span>
      <span class="hljs-built_in">echo</span> <span class="hljs-string">"Failed to create VM: <span class="hljs-variable">$vm_name</span>"</span>
      <span class="hljs-built_in">exit</span> 1
  <span class="hljs-keyword">fi</span>

  <span class="hljs-built_in">echo</span> <span class="hljs-string">"creating vm"</span>
  <span class="hljs-built_in">echo</span> <span class="hljs-variable">$vm_name</span>
}
...
</code></pre>
<p>The <code>run_benchmark()</code> function automates the benchmarking process for a VM by using <strong>local variables</strong> to store the necessary parameters. It retrieves the list of VMs using <code>cudoctl vm list</code> and extracts the VM’s <strong>external IP address</strong> using <code>grep</code> and <code>awk</code>. If no IP is found, the function waits and retries once. If the IP is still not found, it returns 1, indicating a failure.</p>
<pre><code class="lang-bash">...
<span class="hljs-function"><span class="hljs-title">run_benchmark</span></span>() {
  <span class="hljs-built_in">local</span> vm_id=<span class="hljs-variable">$1</span>
  <span class="hljs-built_in">local</span> vcpu=<span class="hljs-variable">$2</span>
  <span class="hljs-built_in">local</span> memory=<span class="hljs-variable">$3</span>
  <span class="hljs-built_in">local</span> disk_size=<span class="hljs-variable">$4</span>

  <span class="hljs-built_in">echo</span> <span class="hljs-string">"Running benchmark on (VM ID: <span class="hljs-variable">$vm_id</span>)..."</span>

  IPS=($(cudoctl vm list | grep -A 2 <span class="hljs-string">"<span class="hljs-variable">$vm_id</span>"</span> | grep externalIP | awk <span class="hljs-string">'{print $2}'</span>))

  <span class="hljs-keyword">if</span> [[ <span class="hljs-variable">${#IPS[@]}</span> -eq 0 ]]; <span class="hljs-keyword">then</span>
    <span class="hljs-built_in">echo</span> <span class="hljs-string">"No IPs found for VM <span class="hljs-variable">$vm_id</span>, waiting 30 seconds and trying again..."</span>
    sleep 30
    IPS=($(cudoctl vm list | grep -A 2 <span class="hljs-string">"<span class="hljs-variable">$vm_id</span>"</span> | grep externalIP | awk <span class="hljs-string">'{print $2}'</span>))

    <span class="hljs-keyword">if</span> [[ <span class="hljs-variable">${#IPS[@]}</span> -eq 0 ]]; <span class="hljs-keyword">then</span>
      <span class="hljs-built_in">echo</span> <span class="hljs-string">"Still no IPs found for VM <span class="hljs-variable">$vm_id</span>"</span>
      <span class="hljs-built_in">return</span> 1
    <span class="hljs-keyword">fi</span>
  <span class="hljs-keyword">fi</span>
...
</code></pre>
<p>If an IP is found, the next part of the <code>run_benchmark()</code> function checks whether SSH access is available. If SSH is accessible, it connects to the VM, updates system packages, installs <code>sysbench</code> (a benchmarking tool), and runs a CPU stress test with four threads. The function extracts the benchmark result and saves it to the results file.</p>
<pre><code class="lang-bash">...
  <span class="hljs-built_in">echo</span> <span class="hljs-string">"Found <span class="hljs-variable">${#IPS[@]}</span> IP(s) for VM <span class="hljs-variable">$vm_id</span>"</span>

  <span class="hljs-keyword">for</span> ip <span class="hljs-keyword">in</span> <span class="hljs-string">"<span class="hljs-variable">${IPS[@]}</span>"</span>; <span class="hljs-keyword">do</span>
    <span class="hljs-built_in">echo</span> <span class="hljs-string">"Attempting to connect to IP: <span class="hljs-variable">$ip</span>"</span>

    <span class="hljs-keyword">if</span> ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no -o BatchMode=yes root@<span class="hljs-variable">$ip</span> <span class="hljs-built_in">echo</span> <span class="hljs-string">"SSH Ready"</span> &amp;&gt;/dev/null; <span class="hljs-keyword">then</span>
      <span class="hljs-built_in">echo</span> <span class="hljs-string">"SSH connection to <span class="hljs-variable">$ip</span> successful"</span>

      OUTPUT=$(ssh -o StrictHostKeyChecking=no -o ConnectTimeout=30 root@<span class="hljs-variable">$ip</span> -v &lt;&lt; <span class="hljs-string">'EOF'</span>
<span class="hljs-built_in">export</span> DEBIAN_FRONTEND=noninteractive
sudo apt-get update -y
sudo apt-get install -y sysbench
sysbench cpu --threads=4 run
EOF
      ) || { <span class="hljs-built_in">echo</span> <span class="hljs-string">"Benchmark failed on <span class="hljs-variable">$ip</span>"</span>; <span class="hljs-built_in">continue</span>; }

      CPU_EVENTS=$(<span class="hljs-built_in">echo</span> <span class="hljs-string">"<span class="hljs-variable">$OUTPUT</span>"</span> | grep <span class="hljs-string">"events per second"</span> | awk <span class="hljs-string">'{print $4}'</span>)

      <span class="hljs-keyword">if</span> [[ -n <span class="hljs-string">"<span class="hljs-variable">$CPU_EVENTS</span>"</span> ]]; <span class="hljs-keyword">then</span>
        <span class="hljs-built_in">echo</span> <span class="hljs-string">"<span class="hljs-variable">$vm_id</span>, <span class="hljs-variable">$vcpu</span>, <span class="hljs-variable">$memory</span>, <span class="hljs-variable">$disk_size</span>, <span class="hljs-variable">$CPU_EVENTS</span>"</span> &gt;&gt; <span class="hljs-variable">$RESULTS_FILE</span>
        <span class="hljs-built_in">echo</span> <span class="hljs-string">"Benchmark completed successfully: <span class="hljs-variable">$CPU_EVENTS</span> events/sec"</span>
        <span class="hljs-built_in">return</span> 0
      <span class="hljs-keyword">else</span>
        <span class="hljs-built_in">echo</span> <span class="hljs-string">"Failed to extract benchmark results from <span class="hljs-variable">$ip</span>"</span>
      <span class="hljs-keyword">fi</span>
    <span class="hljs-keyword">else</span>
      <span class="hljs-built_in">echo</span> <span class="hljs-string">"SSH connection to <span class="hljs-variable">$ip</span> failed, trying next IP if available..."</span>
    <span class="hljs-keyword">fi</span>
  <span class="hljs-keyword">done</span>

  <span class="hljs-built_in">echo</span> <span class="hljs-string">"Failed to run benchmark on any available IP for VM <span class="hljs-variable">$vm_id</span>"</span>
  <span class="hljs-built_in">return</span> 1
}
...
</code></pre>
<p>The <code>destroy_vm()</code> function is responsible for <strong>deleting a virtual machine (VM)</strong>. It takes a single argument, <code>vm_id</code> which represents the unique identifier of the VM, to be deleted. This function helps prevent resource wastage.</p>
<pre><code class="lang-bash">...
<span class="hljs-function"><span class="hljs-title">destroy_vm</span></span>() {
  <span class="hljs-built_in">local</span> vm_id=<span class="hljs-variable">$1</span>
  <span class="hljs-built_in">echo</span> <span class="hljs-string">"Destroying VM: (VM ID: <span class="hljs-variable">$vm_id</span>)..."</span>
  cudoctl vm delete <span class="hljs-string">"<span class="hljs-variable">$vm_id</span>"</span>
}
...
</code></pre>
<p>We iterate through each configuration in the <code>CONFIGS</code> array to create the corresponding VM.</p>
<pre><code class="lang-bash">...
<span class="hljs-keyword">for</span> CONFIG <span class="hljs-keyword">in</span> <span class="hljs-string">"<span class="hljs-variable">${CONFIGS[@]}</span>"</span>; <span class="hljs-keyword">do</span>
  <span class="hljs-built_in">set</span> -- <span class="hljs-variable">$CONFIG</span>
  VM_NAME=<span class="hljs-variable">$1</span>
  DATA_CENTER=<span class="hljs-variable">$2</span>
  MACHINE_TYPE=<span class="hljs-variable">$3</span>
  IMAGE=<span class="hljs-variable">$4</span>
  VCPU=<span class="hljs-variable">$5</span>
  MEMORY=<span class="hljs-variable">$6</span>
  DISK_SIZE=<span class="hljs-variable">$7</span>

  create_vm <span class="hljs-string">"<span class="hljs-variable">$VM_NAME</span>"</span> <span class="hljs-string">"<span class="hljs-variable">$DATA_CENTER</span>"</span> <span class="hljs-string">"<span class="hljs-variable">$MACHINE_TYPE</span>"</span> <span class="hljs-string">"<span class="hljs-variable">$IMAGE</span>"</span> <span class="hljs-string">"<span class="hljs-variable">$VCPU</span>"</span> <span class="hljs-string">"<span class="hljs-variable">$MEMORY</span>"</span> <span class="hljs-string">"<span class="hljs-variable">$DISK_SIZE</span>"</span>
  <span class="hljs-built_in">echo</span> <span class="hljs-string">"Waiting for VM to initialize..."</span>

  <span class="hljs-built_in">echo</span> <span class="hljs-string">"---------------------------------------"</span>
<span class="hljs-keyword">done</span>
...
</code></pre>
<p>We wait <strong>60 seconds</strong> to ensure the VMs have fully booted, preventing benchmarking failures due to uninitialised instances. The SSH known hosts file is cleared to avoid connection mismatches. The script then loops through each configuration in the <code>CONFIGS</code> array, runs the benchmark using the specified parameters, and saves the results. After benchmarking, each VM is deleted to free up resources. Finally, a confirmation message is displayed to indicate successful completion.</p>
<pre><code class="lang-bash">...
sleep 60
<span class="hljs-built_in">echo</span> <span class="hljs-string">''</span> &gt; ~/.ssh/known_hosts

<span class="hljs-keyword">for</span> CONFIG <span class="hljs-keyword">in</span> <span class="hljs-string">"<span class="hljs-variable">${CONFIGS[@]}</span>"</span>; <span class="hljs-keyword">do</span>
  <span class="hljs-built_in">set</span> -- <span class="hljs-variable">$CONFIG</span>
  VM_NAME=<span class="hljs-variable">$1</span>
  VCPU=<span class="hljs-variable">$5</span>
  MEMORY=<span class="hljs-variable">$6</span>
  DISK_SIZE=<span class="hljs-variable">$7</span>

  run_benchmark <span class="hljs-string">"<span class="hljs-variable">$VM_NAME</span>"</span> <span class="hljs-string">"<span class="hljs-variable">$VCPU</span>"</span> <span class="hljs-string">"<span class="hljs-variable">$MEMORY</span>"</span> <span class="hljs-string">"<span class="hljs-variable">$DISK_SIZE</span>"</span>
  destroy_vm <span class="hljs-string">"<span class="hljs-variable">$VM_NAME</span>"</span>
<span class="hljs-keyword">done</span>
<span class="hljs-built_in">echo</span> <span class="hljs-string">"Benchmarking complete. Results saved in <span class="hljs-variable">$RESULTS_FILE</span> and VMs deleted."</span>

<span class="hljs-built_in">exit</span> 0
</code></pre>
<p>To run the entire script using the GitHub gist, run the following:</p>
<pre><code class="lang-bash"><span class="hljs-comment">#save the script</span>
curl -sSL https://gist.github.com/yomaokobiah/5da837f38fe9d1d9423ce55d34a034f9/raw &gt; cudobenchscript.sh
<span class="hljs-comment">#make the script executable</span>
chmod +x cudobenchscript.sh
<span class="hljs-comment">#run the script</span>
./cudobenchscript.sh
</code></pre>
<p>Here’s what it looks like upon successful execution:</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1741617218835/b1c24ca7-7a93-42d9-87ef-25273d79ed66.png" alt class="image--center mx-auto" /></p>
<p>Using the <strong>Cudo Compute CLI</strong> enables you to efficiently manage the entire lifecycle of virtual machines, from creation to benchmarking and deletion. Start experimenting today and unlock the full potential of decentralised cloud computing with <strong>Cudo Compute</strong>!</p>
]]></content:encoded></item><item><title><![CDATA[Email Analysis Using Python 3 (Part II)]]></title><description><![CDATA[Welcome back! This is a sequel to Part I, where we covered making changes to our Gmail account, getting the subject of the email and its sender, and visualizing some of the email data.
The emphasis of this part is on getting the body of the emails. 
...]]></description><link>https://yomaokobiah.com/email-analysis-using-python-3-part-ii</link><guid isPermaLink="true">https://yomaokobiah.com/email-analysis-using-python-3-part-ii</guid><category><![CDATA[email]]></category><category><![CDATA[gmail]]></category><category><![CDATA[Python 3]]></category><dc:creator><![CDATA[Yoma Okobiah]]></dc:creator><pubDate>Tue, 29 Jun 2021 16:22:33 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1624983708655/PCXM6J2jb.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Welcome back! This is a sequel to Part I, where we covered making changes to our Gmail account, getting the subject of the email and its sender, and visualizing some of the email data.
The emphasis of this part is on getting the body of the emails. </p>
<p>If you read the first part and tried it out, I hope it was without hitches. If you did not but are interested in learning how to get the body of the email, you can follow from here. So let's jump right in.</p>
<h2 id="prerequisites">Prerequisites</h2>
<ul>
<li>Python 3</li>
<li>Pandas</li>
<li>A Gmail account</li>
</ul>
<h2 id="getting-the-data">Getting The Data</h2>
<p>This was covered in  Part I, which involves making changes to your Gmail account for IMAPLib to work with it.</p>
<div class="embed-wrapper"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a class="embed-card" href="https://yomaokobiah.com/email-analysis-using-python-3-part-i">https://yomaokobiah.com/email-analysis-using-python-3-part-i</a></div>
<h3 id="step-1-importing-the-required-libraries-to-get-the-email-data">Step 1: Importing the required libraries to get the email data</h3>
<p>Here we import the libraries we need, which are imaplib, email, getpass, and pandas. You may want to install pandas using <code>pip install pandas</code> if you do not have it.</p>
<pre><code><span class="hljs-keyword">import</span> imaplib
<span class="hljs-keyword">import</span> email
<span class="hljs-keyword">import</span> getpass
<span class="hljs-keyword">import</span> pandas as pd
</code></pre><h3 id="step-2-gaining-access-to-the-email-server">Step 2: Gaining access to the email server</h3>
<p>Here we log into the email server with our credentials.</p>
<pre><code>username =  <span class="hljs-keyword">input</span>("Enter the email address: ")
<span class="hljs-keyword">password</span> = getpass.getpass("Enter password: ")
mail = imaplib.IMAP4_SSL(<span class="hljs-string">'imap.gmail.com'</span>)
mail.<span class="hljs-keyword">login</span>(username, <span class="hljs-keyword">password</span>)
</code></pre><h3 id="step-3-specifying-the-mailbox-to-get-data-from">Step 3: Specifying the mailbox to get data from.</h3>
<p>Here we print out the mail list to see the available mailboxes, and we select one.</p>
<pre><code><span class="hljs-selector-tag">print</span>(mail.list())
<span class="hljs-selector-tag">mail</span><span class="hljs-selector-class">.select</span>(<span class="hljs-string">"inbox"</span>)
</code></pre><h3 id="step-4-searching-and-fetching-the-data"><strong>Step 4:</strong> Searching and Fetching the data</h3>
<p>The block of code below searches the selected mailbox with the given criteria fetches the emails and stores them to the variable messages.
Here I am searching for emails from FreeCodeCamp.</p>
<pre><code>result, numbers = mail.<span class="hljs-keyword">search</span>(<span class="hljs-keyword">None</span>, <span class="hljs-string">'(FROM "quincy@freecodecamp.org")'</span>)
uids = numbers[<span class="hljs-number">0</span>].split()
uids = [id.decode("utf-8") <span class="hljs-keyword">for</span> id <span class="hljs-keyword">in</span> uids ]
result, messages = mail.<span class="hljs-keyword">fetch</span>(<span class="hljs-string">','</span>.<span class="hljs-keyword">join</span>(uids) ,<span class="hljs-string">'(RFC822)'</span>)
</code></pre><h3 id="step-5-preparing-the-data-to-be-exported"><strong>Step 5:</strong> Preparing the data to be exported</h3>
<p>The block of code below loops through the fetched emails gets the date it was received, who sent it, the subject of the mail, and the body of the mail. </p>
<ul>
<li>We use the <code>walk()</code> method in the email library to get the parts and subparts of the message. </li>
<li>We use the <code>get_content_type()</code> method to get the email body maintype/subtype.</li>
<li>We use the <code>get_payload()</code> to get a string or message instance of the part. </li>
</ul>
<pre><code>body_list =[]
date_list = []
from_list = [] 
subject_list = []
<span class="hljs-keyword">for</span> _, message in messages[::<span class="hljs-number">2</span>]:
  email_message = email.message_from_bytes(message)
  email_subject = email.header.decode_header(email_message[<span class="hljs-string">'Subject'</span>])[<span class="hljs-number">0</span>]
  <span class="hljs-keyword">for</span> part in email_message.walk():
    <span class="hljs-keyword">if</span> part.get_content_type() == <span class="hljs-string">"text/plain"</span> :
        body = part.get_payload(decode=True)
        body = body.decode(<span class="hljs-string">"utf-8"</span>)
        body_list.<span class="hljs-built_in">append</span>(body)
    <span class="hljs-keyword">else</span>:
        <span class="hljs-keyword">continue</span>
    <span class="hljs-keyword">if</span> isinstance(email_subject[<span class="hljs-number">0</span>],bytes):
      decoded = email_subject.decode(errors=<span class="hljs-string">"ignore"</span>)
      subject_list.<span class="hljs-built_in">append</span>(decoded)
    <span class="hljs-keyword">else</span>:
      subject_list.<span class="hljs-built_in">append</span>(email_subject[<span class="hljs-number">0</span>])
  date_list.<span class="hljs-built_in">append</span>(email_message.get(<span class="hljs-string">'date'</span>))
  fromlist = email_message.get(<span class="hljs-string">'From'</span>)
  fromlist = fromlist.split(<span class="hljs-string">"&lt;"</span>)[<span class="hljs-number">0</span>].replace(<span class="hljs-string">'"'</span>, <span class="hljs-string">''</span>)
  from_list.<span class="hljs-built_in">append</span>(fromlist)
</code></pre><p>Here we convert the objects in date_list to DateTime objects using the <code>to_datetime()</code> method; because the time has its UTC format attached, we sliced off the UTC format.
The retrieved information is then converted to a pandas DataFrame and exported to a CSV file.</p>
<pre><code>date_list = pd.to_datetime(date_list)
date_list = [item.isoformat(<span class="hljs-string">' '</span>)[<span class="hljs-symbol">:-</span><span class="hljs-number">6</span>]<span class="hljs-keyword">for</span> item <span class="hljs-keyword">in</span> date_list]
data = pd.DataFrame(data={<span class="hljs-string">'Date'</span><span class="hljs-symbol">:date_list</span>,<span class="hljs-string">'Sender'</span><span class="hljs-symbol">:from_list</span>,<span class="hljs-string">'Subject'</span><span class="hljs-symbol">:subject_list</span>, <span class="hljs-string">'Body'</span><span class="hljs-symbol">:body_list</span>})
data.to_csv(<span class="hljs-string">'emails.csv'</span>,index=False)
</code></pre><h2 id="data-cleaning">Data Cleaning</h2>
<p>Now we are going to view the data and clean it where necessary to make it readable. But, first, we read in the CSV file and view it: </p>
<pre><code><span class="hljs-keyword">data</span> = pd.read_csv(<span class="hljs-string">"\emails.csv"</span>)
<span class="hljs-keyword">data</span>.head()
</code></pre><p>The output can be seen below, going through there are escape characters in the Body column:
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1624956152300/IGFYoIe_U.png" alt="Screenshot 2021-06-29 at 09.41.50.png" /></p>
<p>The function below removes the escape characters in the text. In this case,<code>"\r\n"</code> as seen in the screenshot above. This makes the text more readable. </p>
<pre><code>def clean_data(<span class="hljs-keyword">data</span>, column, i):
    <span class="hljs-keyword">data</span>.loc[i, column] = <span class="hljs-keyword">data</span>.loc[i, column].split(<span class="hljs-string">"\r\n"</span>)
    new_string = <span class="hljs-string">" "</span>.join(<span class="hljs-keyword">data</span>.loc[i, column])
    new_string = new_string.split(<span class="hljs-string">"'',"</span>)
    <span class="hljs-keyword">data</span>[column][i:i+<span class="hljs-number">1</span>] = pd.DataFrame(<span class="hljs-keyword">data</span> = new_string)
    <span class="hljs-keyword">return</span> <span class="hljs-keyword">data</span>
</code></pre><p>The code below is using the function above to clean every email body in the file.</p>
<pre><code><span class="hljs-keyword">for</span> n <span class="hljs-keyword">in</span> range(len(data)):
    new_data = clean_data(data, column = "Body", i = n)
</code></pre><p>The output can be seen below:
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1624956624943/DkKDceSpT.png" alt="Screenshot 2021-06-29 at 09.50.02.png" /></p>
<p>You are advised to rewrite a new function according to the escape characters you may find in the Subject or Body of the email you retrieved. </p>
<h2 id="conclusion">Conclusion</h2>
<p>I encountered ERRORS while writing this, and the most recurring one was being unable to sign in even after following the instructions on the Google help page. This problem was encountered because I have more than one Gmail account signed in, and I was not using my default email. In case you encounter the same, the solution is outlined below:</p>
<ul>
<li>The instruction said, "If the tips above didn't help, visit https://www.google.com/accounts/DisplayUnlockCaptcha and follow the steps on the page." This opens on a new tab.</li>
<li>The link on the new tab was "https://accounts.google.com/b/0/DisplayUnlockCaptcha" where the digit 0 is for the default account logged in. </li>
<li>Check your accounts in the order in which they are listed and change the digit accordingly (e.g.,”1" is the next email and so on).</li>
</ul>
<p>You can find the full code on GitHub below:</p>
<div class="embed-wrapper"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a class="embed-card" href="https://github.com/yomaokobiah/email_analysis/blob/master/email_analysis_part_II.ipynb">https://github.com/yomaokobiah/email_analysis/blob/master/email_analysis_part_II.ipynb</a></div>
<p>Thank you for getting to the end of this article. I hope you had fun trying it out.</p>
<h3 id="references">References</h3>
<ul>
<li><a target="_blank" href="https://stackoverflow.com/questions/2230037/how-to-fetch-an-email-body-using-imaplib-in-python">Stack Overflow; How to fetch an email body using imaplib in python?</a> </li>
<li><a target="_blank" href="https://docs.python.org/3/library/email.message.html">Email Library Documentation</a></li>
<li><a target="_blank" href="https://docs.python.org/3/library/imaplib.html">IMAP Library Documentation</a></li>
</ul>
]]></content:encoded></item><item><title><![CDATA[Decluttering Mailbox Using Python]]></title><description><![CDATA[I've had so many unread emails in my inbox because they were promotional content, or I was just too lazy to read when they came in. Now imagine opening your mailbox and seeing over a thousand emails — screams in 1917!! How can I declutter as fast as ...]]></description><link>https://yomaokobiah.com/decluttering-mailbox-using-python</link><guid isPermaLink="true">https://yomaokobiah.com/decluttering-mailbox-using-python</guid><category><![CDATA[Python]]></category><category><![CDATA[email]]></category><dc:creator><![CDATA[Yoma Okobiah]]></dc:creator><pubDate>Sat, 29 May 2021 09:28:37 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1622209009365/3EoWBMzGI.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>I've had so many unread emails in my inbox because they were promotional content, or I was just too lazy to read when they came in. Now imagine opening your mailbox and seeing over a thousand emails — screams in 1917!! How can I declutter as fast as possible?</p>
<p>Now I know Gmail has a delete feature, but I believe we're meant to "automate the boring stuff with python," and you may learn something new or not.</p>
<p>In this article, I will show you how to set up a script to send unread/unwanted messages from your inbox to trash using IMAPlib and how to delete unread/unwanted messages using IMAPClient permanently.</p>
<p>Let's jump right into it!</p>
<h2 id="prerequisites">Prerequisites</h2>
<ul>
<li>Python</li>
<li>Gmail account </li>
<li>ImapClient [optional]</li>
</ul>
<h2 id="setting-up-your-gmail-account">Setting up your Gmail account</h2>
<p>IMAP stands for Internet Mail Access protocol. According to Wikipedia, it is "an Internet standard protocol used by email clients to retrieve email messages from a mail server over a TCP/IP connection.”</p>
<p>To get started, we need to set up a Gmail account for the process to be successful. I made the following changes to my Gmail account, which enabled IMAP and turned on less secured apps.</p>
<ol>
<li>First, open Gmail, click on the <strong>settings</strong> ⚙️ icon and click See all settings to enable IMAP.</li>
<li>On the next page, click on the <strong>Forwarding and POP/IMAP</strong> tab.</li>
<li>In the <strong>IMAP Access</strong> section, select <strong>Enable IMAP</strong>. Then click <strong>Save changes</strong>. If you need more help, kindly visit this <a target="_blank" href="https://support.google.com/mail/answer/7126229?hl">Gmail help page</a>.</li>
<li>To turn on less secured apps, navigate to your Google dashboard by clicking on your account avatar in the upper right-hand corner of your screen and then click <strong>My Account</strong> or navigate to myaccount.google.com.</li>
<li>Then choose <strong>Sign-in &amp; security</strong>, scroll down until you see the option Allow less secure apps, and turn the access on.</li>
</ol>
<p>If you still can’t log in after doing the above, kindly visit here for official Google <a target="_blank" href="https://support.google.com/mail/answer/7126229?visit_id=637574811006268631-897992682&amp;rd=2#cantsignin&amp;zippy=%2Ci-cant-sign-in-to-my-email-client">help support</a>. </p>
<h2 id="how-to-trash-your-emails-using-imaplib">How to trash your emails using IMAPlib</h2>
<p>The IMAP library is an inbuilt python library used for accessing and manipulating emails over IMAP. The code below logs you into the desired email address. After logging in, we select the desired mailbox to see a list of all available mailboxes to select from using <code>print(mail.list())</code>.</p>
<h3 id="starter-code-for-imaplib">Starter code for IMAPlib</h3>
<p>The criterion for emails to be sent to the trash in this section are:</p>
<ul>
<li>Unread emails.</li>
<li>Emails based on a specific subject.</li>
<li>Emails based on a specific sender.</li>
</ul>
<pre><code><span class="hljs-keyword">import</span> imaplib
<span class="hljs-keyword">import</span> getpass
username =  <span class="hljs-keyword">input</span>("Enter the email address: ")
<span class="hljs-keyword">password</span> = getpass.getpass("Enter password: ")
mail = imaplib.IMAP4_SSL(<span class="hljs-string">'imap.gmail.com'</span>)
mail.<span class="hljs-keyword">login</span>(username, <span class="hljs-keyword">password</span>)
mail.<span class="hljs-keyword">select</span>("INBOX")
</code></pre><h3 id="trashing-all-unread-emails">Trashing all unread emails</h3>
<p>The code below sends all unread emails to the trash. You never can tell if you need any email, so you have thirty (30) days before Gmail removes it from the trash. First, we search through the emails setting the criteria to "UNSEEN" (which is the flag for unread messages). The variable result is an exit code of the command, while messages are a list that contains an object of type byte. Finally, we loop through the emails in the list and put them in the trash folder.</p>
<pre><code>results, messages = mail.search(<span class="hljs-literal">None</span>, <span class="hljs-string">'UNSEEN'</span>)
messages = messages[<span class="hljs-number">0</span>].split()
<span class="hljs-keyword">for</span> x <span class="hljs-keyword">in</span> messages:
  result, message = mail.store(x, <span class="hljs-string">'+X-GM-LABELS'</span>, <span class="hljs-string">'\\Trash'</span>)
mail.close()
mail.logout()
</code></pre><h3 id="trashing-all-emails-with-a-particular-subject">Trashing all emails with a particular subject</h3>
<p>Here, we search the selected mailbox using our desired criteria. You should replace <code>"Title"</code> with your desired subject in the code below:</p>
<pre><code>results, messages = mail.search(<span class="hljs-literal">None</span>, <span class="hljs-string">'(SUBJECT "Title")'</span>)
messages = messages[<span class="hljs-number">0</span>].split()
<span class="hljs-keyword">for</span> x <span class="hljs-keyword">in</span> messages:
  result, message = mail.store(x, <span class="hljs-string">'+X-GM-LABELS'</span>, <span class="hljs-string">'\\Trash'</span>)
mail.close()
mail.logout()
</code></pre><h3 id="trashing-all-emails-from-a-specific-sender">Trashing all emails from a specific sender</h3>
<p>Here, we search the mailbox using our desired criteria. It would be best if you replaced<code>"abc@xyz.com"</code> with your preferred email address in the code below:</p>
<pre><code>results, messages = mail.search(<span class="hljs-literal">None</span>, <span class="hljs-string">'(FROM "abc@xyz.com")'</span>)
messages = messages[<span class="hljs-number">0</span>].split()
<span class="hljs-keyword">for</span> x <span class="hljs-keyword">in</span> messages:
  result, message = mail.store(x, <span class="hljs-string">'+X-GM-LABELS'</span>, <span class="hljs-string">'\\Trash'</span>)
mail.close()
mail.logout()
</code></pre><p>Here I'll briefly talk about the methods used.</p>
<ul>
<li><code>search()</code> is a method used to look through the selected mailbox, it takes two arguments; a charset and a criterion. The charset can be set to None.</li>
<li><code>store()</code> is a method used to change the flag or label of a message; it takes three arguments; the message, the command, and the flag.</li>
<li><code>close()</code> is a method used to close the selected mailbox.</li>
<li><code>logout()</code> is a method used to close the connection to the server.</li>
<li><code>'+X-GM-LABELS'</code> a label is treated as a folder to make changes to messages using IMAP commands.</li>
</ul>
<h2 id="how-to-permanently-delete-your-emails-using-imapclient">How to permanently delete your emails using IMAPClient</h2>
<p>While researching this article, I came across the IMAPClient library. From its <a target="_blank" href="https://imapclient.readthedocs.io/en/2.2.0/">official documentation</a>, I discovered that: "although IMAPClient actually uses the imaplib module from the Python standard library under the hood, it provides a different API."</p>
<p>Using IMAPClient to delete emails will not send the emails to the trash folder; they will be permanently deleted. Therefore, you should BE SURE you want to delete your emails before using any code snippets here permanently.</p>
<h3 id="starter-code-for-imapclient">Starter Code for IMAPClient</h3>
<p>To use IMAPClient, you have to install the package using the command below:</p>
<pre><code><span class="hljs-attribute">pip</span> install imapclient
</code></pre><p>Next, we import the library and set the server of our email address.</p>
<pre><code><span class="hljs-keyword">from</span> imapclient <span class="hljs-keyword">import</span> IMAPClient
<span class="hljs-keyword">import</span> getpass
<span class="hljs-keyword">server</span> = IMAPClient(<span class="hljs-string">'imap.gmail.com'</span>, use_uid=<span class="hljs-keyword">True</span>)
username =  <span class="hljs-keyword">input</span>("Enter the email address: ")
<span class="hljs-keyword">password</span> = getpass.getpass("Enter password: ")
<span class="hljs-keyword">server</span>.<span class="hljs-keyword">login</span>(username, <span class="hljs-keyword">password</span>)
select_info = <span class="hljs-keyword">server</span>.select_folder("INBOX")
</code></pre><p>The criterion for emails to be sent to the trash in this section are:</p>
<ul>
<li>Unread emails.</li>
<li>Emails based on a specific subject.</li>
<li>Emails based on a specific sender.</li>
</ul>
<h3 id="permanently-deleting-all-unread-emails">Permanently deleting all unread emails</h3>
<p>The code below permanently deletes all unread emails.</p>
<pre><code>messages = server.search(<span class="hljs-string">"UNSEEN"</span>)
for x in messages:
  server.delete_messages(x)
server.logout()
</code></pre><h3 id="permanently-deleting-all-emails-with-a-particular-subject">Permanently deleting all emails with a particular subject</h3>
<p>We search the selected mailbox using our desired criteria. You should replace <code>"Title"</code> with your desired subject in the code below:</p>
<pre><code><span class="hljs-attribute">messages</span> = server.search([<span class="hljs-string">'SUBJECT'</span>, <span class="hljs-string">'Title'</span>])
for x in messages:
  server.delete_messages(x)
server.logout()
</code></pre><h3 id="permanently-deleting-all-emails-from-a-specific-sender">Permanently deleting all emails from a specific sender</h3>
<p>We search the mailbox using our desired criteria. It would be best if you replaced<code>"abc@xyz.com"</code> with your preferred email address in the code below:</p>
<pre><code><span class="hljs-attribute">messages</span> = server.search([<span class="hljs-string">'FROM'</span>, <span class="hljs-string">'abc<span class="hljs-variable">@xyz</span>.com'</span>])
for x in messages:
  server.delete_messages(x)
server.logout()
</code></pre><h3 id="note">Note</h3>
<ul>
<li>Logging out of the server is important</li>
<li>To access all emails while using IMAPlib, use <code>mail.select('"[Gmail]/All Mail"').</code></li>
<li>Ensure you turn off less secure apps.</li>
<li>A lot of emails were harmed in the process of making this tutorial 😂.</li>
</ul>
<h2 id="conclusion">Conclusion</h2>
<p>I had fun writing this, and I hope you did too while reading it. I also encountered ERRORS while writing the script. The most recurring one was being unable to sign in even after following the instructions on the Google help page. This problem was encountered because I have more than one Gmail account signed in, and I was not using my default email. In case you encounter the same, the solution is outlined below:</p>
<ul>
<li>The instruction said, "If the tips above didn't help, visit https://www.google.com/accounts/DisplayUnlockCaptcha and follow the steps on the page." This opens on a new tab.</li>
<li>The link on the new tab was "https://accounts.google.com/b/0/DisplayUnlockCaptcha" where the digit 0 is for the default account logged in. </li>
<li>Check your accounts in the order in which they are listed and change the digit accordingly (e.g.,”1" is the next email and so on).</li>
</ul>
<p>You can find the complete script on GitHub below:</p>
<div class="embed-wrapper"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a class="embed-card" href="https://github.com/yomaokobiah/Decluttering_Email">https://github.com/yomaokobiah/Decluttering_Email</a></div>
<p>Thank you for reading this tutorial!</p>
<h2 id="references">References</h2>
<ul>
<li><a target="_blank" href="https://docs.python.org/3/library/imaplib.html#imaplib.IMAP4.unselect">IMAP Library Documentation</a> </li>
<li><a target="_blank" href="https://gist.github.com/giovaneliberato/b3ebce305262888633c1">Giovane Liberato Github Gist</a> </li>
<li><a target="_blank" href="https://imapclient.readthedocs.io/en/2.2.0/index.html">IMAPClient Documentation</a></li>
</ul>
]]></content:encoded></item><item><title><![CDATA[Arduino Starter Kit Project 03]]></title><description><![CDATA[In  project 02, we wrote our first lines of Arduino code. In this project, we will not only be writing code, but we will also work with our first sensor, the temperature sensor.
Requirements

Three red LEDs
Three 220 ohm resistors
Temperature sensor
...]]></description><link>https://yomaokobiah.com/arduino-starter-kit-project-03</link><guid isPermaLink="true">https://yomaokobiah.com/arduino-starter-kit-project-03</guid><category><![CDATA[arduino]]></category><category><![CDATA[hardware]]></category><dc:creator><![CDATA[Yoma Okobiah]]></dc:creator><pubDate>Mon, 03 Aug 2020 10:46:00 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1596392637080/Enap_F3BH.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>In  <a target="_blank" href="https://yomaokobiah.com/arduino-starter-kit-project-02-ckc3ro1w50036tqs12s0ze78v">project 02</a>, we wrote our first lines of Arduino code. In this project, we will not only be writing code, but we will also work with our first sensor, the temperature sensor.</p>
<h2 id="requirements">Requirements</h2>
<ul>
<li>Three red LEDs</li>
<li>Three 220 ohm resistors</li>
<li>Temperature sensor</li>
<li>Jumper wires</li>
</ul>
<p>The project description as written in the project's book;</p>
<blockquote>
<p> You'll be using a temperature sensor to measure how warm your skin is. This component outputs a changing voltage depending on the temperature it senses. It has three pins: one that connects to the ground, another that connects to power, and a third that outputs a variable voltage to your Arduino. In the sketch for this project, you'll read the sensor's output and use it to turn LEDs on and off, indicating how warm you are. There are several different models of temperature sensors. This model, the TMP36, is convenient because it outputs a voltage that changes directly proportional to the temperature in degrees Celsius. </p>
<p>The Arduino software (IDE) comes with a serial monitor tool that enables you to report back results from the microcontroller. Using the serial monitor, you can get information about the status of the sensors and get an idea about what is happening in your circuit and code as it runs.</p>
</blockquote>
<p>The breadboard and circuit schematics as drawn in the project's book;</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1596382327708/Y5VSjJSvh.png" alt="bread.png" /></p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1596382289208/08Yp8i9Pw.png" alt="circuit.png" /></p>
<h2 id="the-code">The Code</h2>
<p>We covered the structure of Arduino code in project 02, where we talked about the <code>setup()</code> and the <code>loop()</code> functions. We also dealt with variables and digital pins. In this project, in addition, we will be dealing with analog pins and constants.</p>
<p>Analog systems, unlike digital systems, which transmit information in either 0 or 1 format, transmit the information as electric pulses of different amplitude. The Arduino has a built-in Analog-to-Digital Converter (ADC). Analog pins A0-A5 have values that range from 0-1023
Constants are different from variables in the sense that they cannot change.</p>
<p>Let's run through the code below.</p>
<ul>
<li>We start by defining our constants. The sensorPin is the analog pin on which the sensor is connected. The baselineTemp is the temperature in your current environment. If you have no means of measuring the temperature, let's give it the default value for room temperature, 25 degrees Celsius.</li>
<li>In the <code>setup()</code>, we use <code>Serial.begin()</code> to allow the Arduino to send messages via the Serial monitor. It takes two arguments <code>Serial.begin(speed, config)</code>, the latter is optional. In this case, we're telling the Arduino to communicate at a rate of 9600 bits per second.</li>
<li>We use a <code>for()</code> loop to set the pins we connected to the LEDs, unlike project 02, where we defined each pin one after the other, here we set everything in the loop LOW, pin 2 to 4.</li>
<li>In the <code>loop()</code>, we defined a variable sensorVal which stores the value of the sensorPin, which is our analog pin. Then, we use <code>analogRead()</code> to get the value from the analog pin.</li>
<li><code>Serial.print()</code> displays human-readable data on the serial monitor. It can display the value of a variable or of anything you type in quotation marks.</li>
<li>Here's a formula for converting ADC reading to voltage. 
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1596388588233/-jTxn-svL.png" alt="Screenshot 2020-08-02 at 18.13.21.png" /></li>
</ul>
<p>In this case, we're looking for the voltage value, making that the subject of the formula, we store the value in the variable named voltage. Then we print it out to the serial monitor.</p>
<ul>
<li>We then convert the voltage to temperature(Celsius) using the formula "(voltage - .5) * 100", then we print this value to the serial monitor using the <code>Serial.println()</code> this is different from the <code>Serial.print()</code> as it creates a new line.</li>
<li>We then use the temperature to set up our if and else statements. If the temperature is less than our baselineTemp incremented by 2, we set all three LEDs off. Then, we start turning each LED on as the temperature rises.</li>
<li>We set a delay so we can easily read the values on the serial monitor.</li>
</ul>
<p>The code as written in the project's book:</p>
<pre><code><span class="hljs-keyword">const</span> <span class="hljs-keyword">int</span> sensorPin = A0;
<span class="hljs-keyword">const</span> <span class="hljs-keyword">float</span> baselineTemp = <span class="hljs-number">25.00</span>;
<span class="hljs-keyword">void</span> setup() {
  <span class="hljs-comment">// put your setup code here, to run once:</span>
  Serial.begin(<span class="hljs-number">9600</span>);
  <span class="hljs-keyword">for</span>(<span class="hljs-keyword">int</span> pinNumber=<span class="hljs-number">2</span>; pinNumber &lt;<span class="hljs-number">5</span>; pinNumber++){
    pinMode(pinNumber, OUTPUT);
    digitalWrite(pinNumber, LOW);
  }
}

<span class="hljs-keyword">void</span> loop() {
  <span class="hljs-comment">// put your main code here, to run repeatedly:</span>
  <span class="hljs-keyword">int</span> sensorVal = analogRead(sensorPin);
  Serial.<span class="hljs-keyword">print</span>(<span class="hljs-string">"Sensor Value: "</span>);
  Serial.<span class="hljs-keyword">print</span>(sensorVal);

  <span class="hljs-comment">// convert ADC reading to Voltage</span>
  <span class="hljs-keyword">float</span> voltage = (sensorVal/<span class="hljs-number">1024.0</span>) * <span class="hljs-number">5.0</span>;
  Serial.<span class="hljs-keyword">print</span>(<span class="hljs-string">", Volts: "</span>);
  Serial.<span class="hljs-keyword">print</span>(voltage);
  Serial.<span class="hljs-keyword">print</span>(<span class="hljs-string">", degrees C: "</span>);

  <span class="hljs-comment">// convert the voltage to temperature in degrees</span>
  <span class="hljs-keyword">float</span> temperature = (voltage - <span class="hljs-number">0.5</span>) * <span class="hljs-number">100</span>;
  Serial.println(temperature);
  <span class="hljs-keyword">if</span>(temperature &lt; baselineTemp+<span class="hljs-number">2</span>){
    digitalWrite(<span class="hljs-number">2</span>, LOW);
    digitalWrite(<span class="hljs-number">3</span>, LOW);
    digitalWrite(<span class="hljs-number">4</span>, LOW);
  }
  <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span>(temperature &gt;= baselineTemp+<span class="hljs-number">2</span> &amp;&amp; temperature &lt; baselineTemp+<span class="hljs-number">4</span>){
    digitalWrite(<span class="hljs-number">2</span>, HIGH);
    digitalWrite(<span class="hljs-number">3</span>, LOW);
    digitalWrite(<span class="hljs-number">4</span>, LOW);
  }
  <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span>(temperature &gt;= baselineTemp+<span class="hljs-number">4</span> &amp;&amp; temperature &lt; baselineTemp+<span class="hljs-number">6</span>){
    digitalWrite(<span class="hljs-number">2</span>, HIGH);
    digitalWrite(<span class="hljs-number">3</span>, HIGH);
    digitalWrite(<span class="hljs-number">4</span>, LOW);
  }
  <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span>(temperature &gt;= baselineTemp+<span class="hljs-number">6</span>){
    digitalWrite(<span class="hljs-number">2</span>, HIGH);
    digitalWrite(<span class="hljs-number">3</span>, HIGH);
    digitalWrite(<span class="hljs-number">4</span>, HIGH);}
  delay(<span class="hljs-number">1</span>);
}
</code></pre><p>The steps to upload the program to your board;</p>
<ul>
<li>Save the sketch</li>
<li><p>From the menubar, click tools;</p>
<ul>
<li>Make sure you have the right board selected</li>
<li>Select a port</li>
<li>Set the programmer to ArduinoISP</li>
</ul>
</li>
<li>The upload button is the right arrow.</li>
</ul>
<p>Let's check out the Serial monitor:</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1596391016420/3D1LSdseO.png" alt="Screenshot 2020-08-02 at 18.33.25.png" /></p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1596391442390/a_BhsxRCT.png" alt="Screenshot 2020-08-02 at 19.03.30.png" /></p>
<p>If you weren't sure of your baselineTemp before you touch the sensor, the temperature reading is that of your environment, and you can update your baselineTemp and upload the code again.</p>
<p>Your body temperature is always higher than your environment; if you hold the sensor for a while, one or more LEDs should light up. </p>
<p>My body temperature was hot enough to make one LED light up.
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1596392516696/ZVnSm2v7J.gif" alt="20200802_191149_1.gif" /></p>
<h2 id="using-lcd">Using LCD</h2>
<p>I decided to use a 16x2 LCD to display the temperature and a Potentiometer to control the contrast on the LCD screen.</p>
<h3 id="requirements">Requirements</h3>
<ul>
<li>One potentiometer (optional)</li>
<li>One 220 ohm resistor</li>
<li>Temperature sensor</li>
<li>16x2 LCD</li>
<li>Jumper wires</li>
</ul>
<h3 id="schematics">Schematics</h3>
<p>Something to note:
The LCD is labeled from 1 to 16. In this diagram, one is at the bottom and 16 at the top.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1596404340463/LwlpVh9Res.png" alt="Screenshot 2020-08-02 at 22.37.55.png" /></p>
<h3 id="the-code">The Code</h3>
<p>Let's jump into the differences here.</p>
<ul>
<li>Here, we import the LiquidCrystal library and initialize it by telling it what pins to use to communicate. We don't need the baselineTemp here.</li>
<li>We use the <code>lcd.print()</code> to display content to the LCD screen. It works the same way as <code>Serial.print()</code>; the difference is that we're displaying to the LCD and the Serial Monitor, respectively.</li>
<li>Then, we delay for 2 seconds to keep the LCD from flashing the values the way the values flash in the Serial Monitor. Finally, we use the <code>lcd.clear()</code> so that each value stands alone on the screen.</li>
</ul>
<pre><code><span class="hljs-comment">#include &lt;LiquidCrystal.h&gt;</span>
LiquidCrystal lcd(<span class="hljs-number">12</span>,<span class="hljs-number">11</span>,<span class="hljs-number">5</span>,<span class="hljs-number">4</span>,<span class="hljs-number">3</span>,<span class="hljs-number">2</span>);
<span class="hljs-keyword">const</span> <span class="hljs-keyword">int</span> sensorPin = A0;
<span class="hljs-keyword">void</span> setup() {
  <span class="hljs-comment">// put your setup code here, to run once:</span>
  Serial.begin(<span class="hljs-number">9600</span>);
  lcd.begin(<span class="hljs-number">16</span>,<span class="hljs-number">2</span>);
  }

<span class="hljs-keyword">void</span> loop() {
  <span class="hljs-comment">// put your main code here, to run repeatedly:</span>
  <span class="hljs-keyword">int</span> sensorVal = analogRead(sensorPin);
  Serial.<span class="hljs-keyword">print</span>(<span class="hljs-string">"Sensor Value: "</span>);
  Serial.<span class="hljs-keyword">print</span>(sensorVal);

  <span class="hljs-keyword">float</span> voltage = (sensorVal/<span class="hljs-number">1024.0</span>) * <span class="hljs-number">5.0</span>;
  Serial.<span class="hljs-keyword">print</span>(<span class="hljs-string">", Volts: "</span>);
  Serial.<span class="hljs-keyword">print</span>(voltage);
  Serial.<span class="hljs-keyword">print</span>(<span class="hljs-string">", degrees C: "</span>);

  <span class="hljs-keyword">float</span> temperature = (voltage - <span class="hljs-number">.5</span>) * <span class="hljs-number">100</span>;
  Serial.println(temperature);

  lcd.<span class="hljs-keyword">print</span>(temperature);
  lcd.<span class="hljs-keyword">print</span>(<span class="hljs-string">" degrees C"</span>);
  delay(<span class="hljs-number">2000</span>); <span class="hljs-comment">//two seconds delay</span>

  lcd.clear();
  delay(<span class="hljs-number">50</span>);
}
</code></pre><p>After uploading your program, the temperature you see on display is the temperature of the environment. Place your finger over the sensor and observe the changes in the temperature values displayed on the screen.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1596406721331/bSzj_9L-X.gif" alt="20200802_231418_1.gif" /></p>
<h3 id="references">References</h3>
<p><a target="_blank" href="https://store.arduino.cc/genuino-starter-kit">Arduino Projects Book</a></p>
<p><a target="_blank" href="https://create.arduino.cc/projecthub/Parry21/temperature-on-an-lcd-d2e9d3">Temperature on an LCD</a></p>
]]></content:encoded></item><item><title><![CDATA[Arduino Starter Kit Project 02]]></title><description><![CDATA[I am particularly excited about project 02 not only because it was named Spaceship Interface but also because this is the first project where we will write code for the Arduino Uno board.
If you haven't gotten your Arduino IDE installed yet follow th...]]></description><link>https://yomaokobiah.com/arduino-starter-kit-project-02</link><guid isPermaLink="true">https://yomaokobiah.com/arduino-starter-kit-project-02</guid><category><![CDATA[arduino]]></category><category><![CDATA[hardware]]></category><dc:creator><![CDATA[Yoma Okobiah]]></dc:creator><pubDate>Wed, 01 Jul 2020 19:42:20 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1593632448698/EVoq9J0Fz.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>I am particularly excited about project 02 not only because it was named Spaceship Interface but also because this is the first project where we will write code for the Arduino Uno board.</p>
<p>If you haven&#39;t gotten your Arduino IDE installed yet follow the  <a target='_blank' rel='noopener noreferrer'  href="https://www.arduino.cc/en/Guide/HomePage">official guide</a>.</p>
<h2 id="requirements">Requirements</h2>
<ul>
<li>Three LEDs (1 green, 2 red)</li>
<li>Three 220 ohm resistor</li>
<li>10 kilohm resistor</li>
<li>Switch</li>
</ul>
<p>The description for project 02 is as follows;</p>
<blockquote>
<p>You&#39;ll be building something that could have been a spaceship interface in a 1970&#39;s science fiction movie. You&#39;ll make a cool control panel with a switch and lights that turn on when you press the switch. You can decide whether the lights mean &quot;Engage Hyperdrive&quot; or &quot;Fire the lasers!&quot;. A green LED will be on, until you press a button. When the Arduino gets a signal from the button, the green light will turn off and 2 other lights will start blinking.</p>
</blockquote>
<p>Here are a few things to note as you make connections to your breadboard;</p>
<ul>
<li>The longer leg of the LED is the anode (+ve) while the shorter leg is the cathode (-ve).</li>
<li>The button should be placed at the center of the breadboard.</li>
</ul>
<p>Here&#39;s a view of the breadboard and Uno</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1593564374992/3knVjQhRE.png" alt="Screenshot 2020-07-01 at 01.33.23.png"></p>
<p>Here&#39;s a schematic of the circuit
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1593564393801/CV2u5f__I.png" alt="Screenshot 2020-07-01 at 01.29.06.png"></p>
<p>I took a course in school in my 5th semester called Digital Electronics and Logic Design, in that course I learnt about system states of digital systems which could be LOW or HIGH, INPUT or OUTPUT, 0 or 1.</p>
<p>Each program in the Arduino IDE is called a sketch. A sketch has two required functions for it to work the <code>setup()</code> and the <code>loop()</code>. </p>
<p>Before declaring the functions we define variables, the syntax seems similar to C++ (I say C++ because that&#39;s a language I recently started learning) where we define variables by <code>variable_type variable_name = variable;</code></p>
<p>The <code>setup()</code> function runs only once, here we design the state of the digital pins to be inputs or outputs using the <code>pinMode()</code> function. In this case our button is input.</p>
<p>The <code>loop()</code> function runs continuously as the name implies. The loop is where the bulk of the program lies, this is where we check for;</p>
<ul>
<li>The voltage of the input using the <code>digitalRead()</code> function which checks the chosen pin passed as an argument for voltage. In this case we store it in the switchState variable.</li>
<li>If there is a voltage on the pin, switchState will have a value of 1 which is HIGH, if there is no voltage switchState will have a value of 0 which is LOW. If switchState is LOW it means the button</li>
<li>We use the <code>digitalWrite()</code> function to send voltage to a pin either HIGH or LOW. HIGH would be the voltage we have selected on our board (5V in this case) and LOW would be 0V. <code>digitalWrite()</code> takes two arguments, the pin and what value to give it.</li>
<li>The else statement is there if switchState is HIGH, then we set the green LED and one red LED to LOW, we delay and toggle the red LEDs.</li>
</ul>
<p>The code we&#39;ll be uploading to our Board;</p>
<pre><code><span class="hljs-keyword">int</span> switchState = <span class="hljs-number">0</span>;
<span class="hljs-function"><span class="hljs-keyword">void</span> <span class="hljs-title">setup</span><span class="hljs-params">()</span> </span>{
  <span class="hljs-comment">// put your setup code here, to run once:</span>
  pinMode(<span class="hljs-number">3</span>, OUTPUT);
  pinMode(<span class="hljs-number">4</span>, OUTPUT);
  pinMode(<span class="hljs-number">5</span>, OUTPUT);
  pinMode(<span class="hljs-number">2</span>, INPUT);
}

<span class="hljs-function"><span class="hljs-keyword">void</span> <span class="hljs-title">loop</span><span class="hljs-params">()</span> </span>{
  <span class="hljs-comment">// put your main code here, to run repeatedly:</span>
  switchState = digitalRead(<span class="hljs-number">2</span>);
  <span class="hljs-keyword">if</span> (switchState == LOW ) {
    <span class="hljs-comment">// the button is not pressed</span>
    digitalWrite(<span class="hljs-number">3</span>, HIGH);
    digitalWrite(<span class="hljs-number">4</span>, LOW);
    digitalWrite(<span class="hljs-number">5</span>, LOW);}
   <span class="hljs-keyword">else</span> {
      <span class="hljs-comment">// the button is pressed</span>
      digitalWrite(<span class="hljs-number">3</span>, LOW);
      digitalWrite(<span class="hljs-number">4</span>, LOW);
      digitalWrite(<span class="hljs-number">5</span>, HIGH);

      delay(<span class="hljs-number">250</span>);
      digitalWrite(<span class="hljs-number">4</span>, HIGH);
      digitalWrite(<span class="hljs-number">5</span>, LOW);
      delay(<span class="hljs-number">250</span>);
    }
}
</code></pre><p>The steps to upload the program to your board;</p>
<ul>
<li>Save your sketch</li>
<li><p>From the menubar, click tools;</p>
<ul>
<li>Make sure you have the right board selected</li>
<li>Select a port</li>
<li>Set the programmer to ArduinoISP</li>
</ul>
</li>
<li>The upload button is the right arrow.</li>
</ul>
<p>The output:</p>
<p>The cover below came with the Arduino Starter Kit.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1593619763900/dlPECab_S.gif" alt="20200701_155802_1.gif"></p>
<h3 id="references">References</h3>
<ul>
<li>Arduino Projects Book</li>
</ul>
]]></content:encoded></item><item><title><![CDATA[Arduino Starter Kit Project 01]]></title><description><![CDATA[I was super excited to receive my Arduino starter kit (I call it ASKit) and Engineering kit from the official Arduino store. I jumped straight into exploring the components in the Arduino starter kit, I'm yet to open the engineering kit. It came with...]]></description><link>https://yomaokobiah.com/arduino-starter-kit-project-01</link><guid isPermaLink="true">https://yomaokobiah.com/arduino-starter-kit-project-01</guid><category><![CDATA[arduino]]></category><category><![CDATA[hardware]]></category><dc:creator><![CDATA[Yoma Okobiah]]></dc:creator><pubDate>Mon, 29 Jun 2020 21:55:21 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1593467519817/4LyhhYmyz.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>I was super excited to receive my Arduino starter kit (I call it ASKit) and Engineering kit from the official Arduino store. I jumped straight into exploring the components in the Arduino starter kit, I&#39;m yet to open the engineering kit. It came with a cool projects book and in this article I&#39;ll be sharing a few things about project 01 which covered turning LEDs on.</p>
<p>The Arduino starter kit came with the Arduino Uno, a breadboard, a base and loads of other cool components. I particularly love the breadboard as it means no soldering and all components are reusable and the circuit is temporary.</p>
<h2 id="requirements">Requirements</h2>
<ul>
<li>220 ohm Resistor</li>
<li>LED</li>
<li>Two push button switches</li>
<li>5V power supply</li>
</ul>
<h2 id="without-a-push-button-">Without a push button.</h2>
<p>Here&#39;s the schematic circuit diagram below;</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1593463103085/VQJhwQTUW.png" alt="Screenshot 2020-06-29 at 21.36.14.png"></p>
<p>The Uno has a 5V power supply pin and a 3.3V pin, I used the 5V power supply pin and connected the Uno to my laptop with a cable.
The longer leg of the LED is the anode (+end) and the shorter leg is the cathode (-end).</p>
<p>Here&#39;s how it looks;</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1593463960988/QFYjl8ei2.jpeg" alt="breadboard_1.jpg"></p>
<h2 id="with-one-push-button-">With one push button.</h2>
<p>Here&#39;s the schematic circuit diagram below;</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1593464929731/_aaHB38tT.png" alt="Screenshot 2020-06-29 at 21.00.05.png"></p>
<p><em>Note: The button is placed at the center of the breadboard with one part at the other side of the ravine to keep the horizontal part on the same row.</em></p>
<p>On the left hand side the button hasn&#39;t been pressed hence the LED is still off, and on the right hand side the button is pressed, so the LED is on.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1593465299686/RB-yq5a8a.jpeg" alt="one_button.jpg"></p>
<h2 id="with-two-push-buttons-serial-circuit-">With two push buttons, serial circuit.</h2>
<p>Here&#39;s the schematic circuit diagram below;
<img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1593466155729/LEDlZjQTJ.png" alt="Screenshot 2020-06-29 at 22.19.58.png"></p>
<p>On the left hand side no button is pressed hence the LED is off. On the right hand side both buttons have to be pressed for the LED to come on because they&#39;re connected in series and the same current flows through them.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1593466246785/fTJIX_RMM.jpeg" alt="two_button.jpg"></p>
<h2 id="with-two-push-buttons-parallel-circuit-">With two push buttons, parallel circuit.</h2>
<p>Here&#39;s the schematic circuit diagram below;</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1593466320461/OL-kVVneF.png" alt="Screenshot 2020-06-29 at 22.21.16.png"></p>
<p>On the left hand side no button is pressed hence the LED is off. On the right hand either button can be pressed for the LED to come on because they&#39;re connected in parallel and the current is split between both of them.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1593466416332/U5yXTms7F.jpeg" alt="two_button_parallel.jpg"></p>
<p>One exciting part of junior secondary school was learning about Ohms law in Integrated Science class which states that:</p>
<blockquote>
<p> The current through a conductor between two points is directly proportional to the voltage across the two points.</p>
</blockquote>
<p>Hence V = IR, using ohms law to calculate the current passing through the LED, where V = 5 and R = 220, we have approximately 0.02 amps. We used the 220 ohm resistor because 0.02 amps is just about the maximum current for common LEDs.</p>
<p>When you explore further, note;</p>
<ul>
<li>Always turn off your power supply before making connections.</li>
<li>If you want to use two LEDs do not connect them in parallel to one resistor.</li>
</ul>
<h3 id="references-">References:</h3>
<ul>
<li>Arduino Projects Book</li>
</ul>
]]></content:encoded></item><item><title><![CDATA[Email Analysis Using Python 3 (Part I)]]></title><description><![CDATA[There is a lot of data out there, mostly unstructured. For example, emails are a great source of communication data; there is no limit to what we can harness from them.
At the end of this tutorial, you would be able to get email data for insights.
Pr...]]></description><link>https://yomaokobiah.com/email-analysis-using-python-3-part-i</link><guid isPermaLink="true">https://yomaokobiah.com/email-analysis-using-python-3-part-i</guid><category><![CDATA[Python 3]]></category><category><![CDATA[data analysis]]></category><category><![CDATA[email]]></category><category><![CDATA[Data Science]]></category><dc:creator><![CDATA[Yoma Okobiah]]></dc:creator><pubDate>Fri, 29 May 2020 12:51:57 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1590775549500/trCSN_fn6.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>There is a lot of data out there, mostly unstructured. For example, emails are a great source of communication data; there is no limit to what we can harness from them.
At the end of this tutorial, you would be able to get email data for insights.</p>
<h2 id="prerequisites">Prerequisites</h2>
<ul>
<li>Python 3</li>
<li>Pandas</li>
<li>Matplotlib</li>
<li>Seaborn</li>
<li>Wordcloud</li>
<li>A Gmail account</li>
</ul>
<h2 id="getting-the-data">Getting The Data</h2>
<p>There are several ways to achieve the aim of this article; find below how I did mine.</p>
<p>Here a Gmail account is being used; for the imaplib script to work, I made the following changes to my account; enabled IMAP and turned on less secured apps.</p>
<ol>
<li>First, open Gmail, click on the <strong>settings</strong> ⚙️ icon and click See all settings to enable IMAP.</li>
<li>On the next page, click on the <strong>Forwarding and POP/IMAP</strong> tab.</li>
<li>In the <strong>IMAP Access</strong> section, select <strong>Enable IMAP</strong>. Then click <strong>Save changes</strong>. If you need more help, kindly visit this <a target="_blank" href="https://support.google.com/mail/answer/7126229?hl">Gmail help page</a>.</li>
<li>To turn on less secured apps, navigate to your Google dashboard by clicking on your account avatar in the upper right-hand corner of your screen and then click <strong>My Account</strong> or navigate to myaccount.google.com.</li>
<li>Then choose <strong>Sign-in &amp; security</strong>, scroll down until you see the option Allow less secure apps, and turn the access on.</li>
</ol>
<p>If you still can’t log in after doing the above, kindly visit here for official Google <a target="_blank" href="https://support.google.com/mail/answer/7126229?visit_id=637574811006268631-897992682&amp;rd=2#cantsignin&amp;zippy=%2Ci-cant-sign-in-to-my-email-client">help support</a>. </p>
<h3 id="step-1-importing-the-required-libraries-to-get-the-email-data">Step 1: Importing the required libraries to get the email data</h3>
<ul>
<li><p>imaplib is an Internet Message Access Protocol (IMAP) library</p>
</li>
<li><p>email is a python library that parses, handles, and generates email messages.</p>
</li>
<li><p>getpass is a python library that contains utilities to get a password or current username</p>
</li>
<li><p>pandas is a python library for data manipulation and analysis.</p>
</li>
</ul>
<pre><code><span class="hljs-keyword">import</span> imaplib
<span class="hljs-keyword">import</span> email
<span class="hljs-keyword">import</span> getpass
<span class="hljs-keyword">import</span> pandas as pd
</code></pre><h3 id="step-2-gaining-access-to-the-email-server">Step 2: Gaining access to the email server</h3>
<ul>
<li><em>username</em> is the email address. </li>
<li><em>password</em> is the password to the email address when prompted. [If you don't want to use the getpass package, you can enter your password as a string.]</li>
<li><em>mail</em> is the email server we're going to connect to, and it varies; for this tutorial, we're using Gmail.</li>
<li><em>mail.login</em> logs into the server using the provided credentials.</li>
</ul>
<pre><code>username =  <span class="hljs-keyword">input</span>("Enter the email address: ")
<span class="hljs-keyword">password</span> = getpass.getpass("Enter password: ")
mail = imaplib.IMAP4_SSL(<span class="hljs-string">'imap.gmail.com'</span>)
mail.<span class="hljs-keyword">login</span>(username, <span class="hljs-keyword">password</span>)
</code></pre><h3 id="step-3-specifying-the-mailbox-to-get-data-from">Step 3: Specifying the mailbox to get data from.</h3>
<ul>
<li><code>mail.list()</code> is a method that gives a list of the mailboxes - i.e., inbox, draft, and so on in the email address.</li>
<li><code>mail.select()</code> is a method that takes an argument of the mailbox you want to get data from.</li>
</ul>
<pre><code><span class="hljs-selector-tag">print</span>(mail.list())
<span class="hljs-selector-tag">mail</span><span class="hljs-selector-class">.select</span>(<span class="hljs-string">"inbox"</span>)
</code></pre><h3 id="step-4-searching-and-fetching-the-data">Step 4: Searching and Fetching the data</h3>
<ul>
<li><strong>Line 1:</strong> mail.uid() is a method whose first argument is the command you want to execute; in this case, the command is "search." The rest of the arguments are used for the search. (Search gives from oldest to recent)</li>
<li><strong>Line 1:</strong> result is an exit code of the command while numbers is a list that contains an object of type byte.</li>
<li><strong>Line 2:</strong> is a list of every section in numbers.</li>
<li><strong>Line 3:</strong> is a list of decoded bytes</li>
<li><strong>Line 4:</strong> is a slice of the recent 100 items (recall that search orders it from oldest to recent). </li>
<li><strong>Line 5:</strong> the command we want to execute is "fetch" and store it in messages. We're fetching the subject of the messages in this case.</li>
</ul>
<pre><code>result, numbers = mail.uid(<span class="hljs-string">'search'</span>, <span class="hljs-keyword">None</span>, "ALL")
uids = numbers[<span class="hljs-number">0</span>].split()
uids = [id.decode("utf-8") <span class="hljs-keyword">for</span> id <span class="hljs-keyword">in</span> uids ]
uids = uids[<span class="hljs-number">-1</span>:<span class="hljs-number">-101</span>:<span class="hljs-number">-1</span>]
result, messages = mail.uid(<span class="hljs-string">'fetch'</span>, <span class="hljs-string">','</span>.<span class="hljs-keyword">join</span>(uids), <span class="hljs-string">'(BODY[HEADER.FIELDS (SUBJECT FROM DATE)])'</span>)
</code></pre><h3 id="step-5-preparing-the-data-to-be-exported">Step 5: Preparing the data to be exported</h3>
<ul>
<li><strong>Line 1-3:</strong> empty lists for the data we specified in messages.</li>
<li><strong>Line 4:</strong> looping through the content of the message we fetched. Using a step of two because it returned a tuple of two items.</li>
<li><strong>Line 5:</strong> parsing the bytes email to message object.</li>
<li><strong>Line 6-11:</strong> msg is in bytes; to use it, it had to be decoded to a format we can read.</li>
<li><strong>Line 12:</strong> adding the dates to date_list.</li>
<li><strong>Line 13-15:</strong> getting the sender detail, it's in the format "Sender name"  hence the split and replace methods are used to get only the "Sender name".</li>
<li><strong>Line 16-19:</strong> converting the objects in date_list to DateTime objects, because the time has its UTC format attached, a new list was created, and the UTC format was sliced off from each object in the list.</li>
<li><strong>Line 20-22:</strong> checking the length of created lists because arrays have to be the same length.</li>
<li><strong>Line 23-25:</strong> converting the lists to a dictionary and then a pandas data frame, viewing it, and saving it for download.</li>
</ul>
<pre><code>date_list = []
from_list = [] 
subject_text = []
<span class="hljs-keyword">for</span> i, message in messages[::<span class="hljs-number">2</span>]:
    msg = email.message_from_bytes(message)
    decode = email.header.decode_header(msg[<span class="hljs-string">'Subject'</span>])[<span class="hljs-number">0</span>]
    <span class="hljs-keyword">if</span> isinstance(decode[<span class="hljs-number">0</span>],bytes):
        decoded = decode[<span class="hljs-number">0</span>].decode()
        subject_text.<span class="hljs-built_in">append</span>(decoded)
    <span class="hljs-keyword">else</span>:
        subject_text.<span class="hljs-built_in">append</span>(decode[<span class="hljs-number">0</span>])
    date_list.<span class="hljs-built_in">append</span>(msg.get(<span class="hljs-string">'date'</span>))
    fromlist = msg.get(<span class="hljs-string">'From'</span>)
    fromlist = fromlist.split(<span class="hljs-string">"&lt;"</span>)[<span class="hljs-number">0</span>].replace(<span class="hljs-string">'"'</span>, <span class="hljs-string">''</span>)
    from_list1.<span class="hljs-built_in">append</span>(fromlist)
date_list = pd.to_datetime(date_list)
date_list1 = []
<span class="hljs-keyword">for</span> item in date_list:
    date_list1.<span class="hljs-built_in">append</span>(item.isoformat(<span class="hljs-string">' '</span>)[:<span class="hljs-number">-6</span>])
<span class="hljs-built_in">print</span>(<span class="hljs-built_in">len</span>(subject_text))
<span class="hljs-built_in">print</span>(<span class="hljs-built_in">len</span>(from_list))
<span class="hljs-built_in">print</span>(<span class="hljs-built_in">len</span>(date_list1))
df = pd.DataFrame(data={<span class="hljs-string">'Date'</span>:date_list1, <span class="hljs-string">'Sender'</span>:from_list, <span class="hljs-string">'Subject'</span>:subject_text})
<span class="hljs-built_in">print</span>(df.head())
df.to_csv(<span class="hljs-string">'inbox_email.csv'</span>,index=False)
</code></pre><h2 id="visualisation">Visualisation</h2>
<p>Now that we have the email data in CSV format, we can read it using pandas and visualize it.
There are several Python data visualization libraries, but here I used Wordcloud, Matplotlib, and Seaborn. I wanted to see an infographic on the most used words in the subjects of my emails, and here is how I did it.</p>
<h3 id="step-1-reading-and-viewing-the-csv">Step 1: Reading and viewing the CSV</h3>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1590503891198/tEDPANCq-.png" alt="read.png" /></p>
<h3 id="step-2-getting-statistical-data">Step 2: Getting statistical data</h3>
<p>I used the describe method to get the statistical data, unique values, and insight into the data.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1590503684511/0UnaE1iPk.png" alt="describee.png" /></p>
<h3 id="step-3-creating-new-variables">Step 3: Creating new variables</h3>
<p>I created two variables; Time and SinceMid. SinceMid is the number of hours after midnight.</p>
<blockquote>
<p><em>(Note: The time can be deleted from the date column)</em></p>
</blockquote>
<pre><code><span class="hljs-keyword">from</span> DateTime <span class="hljs-keyword">import</span> datetime
FMT = <span class="hljs-string">'%H:%M:%S'</span>
emails[<span class="hljs-string">'Time'</span>] = emails[<span class="hljs-string">'Date'</span>].apply(<span class="hljs-keyword">lambda</span> x: datetime.strptime(x, <span class="hljs-string">'%Y-%m-%-d%H:%M:%S'</span>).strftime(FMT))
emails[<span class="hljs-string">'SinceMid'</span>] = emails[<span class="hljs-string">'Time'</span>].apply(<span class="hljs-keyword">lambda</span> x: (datetime.strptime(x, FMT) - datetime.strptime(<span class="hljs-string">"00:00:00"</span>, FMT)).seconds) / <span class="hljs-number">60</span> / <span class="hljs-number">60</span>
</code></pre><p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1590539232938/4AdcGk2Uf.png" alt="time.png" /></p>
<h3 id="step-4-the-plots">Step 4: The plots</h3>
<p>I created a wordcloud image of the most used words in the subjects of my emails. In this example, there are no stopwords; stopwords are usually filtered out like they're usually not informative.</p>
<pre><code><span class="hljs-keyword">from</span> wordcloud <span class="hljs-keyword">import</span> WordCloud
<span class="hljs-keyword">import</span> matplotlib.pyplot <span class="hljs-keyword">as</span> plt


<span class="hljs-comment"># Create a list of words</span>
text = <span class="hljs-string">""</span>
<span class="hljs-keyword">for</span> item <span class="hljs-keyword">in</span> emails[<span class="hljs-string">"Subject"</span>]:
    <span class="hljs-keyword">if</span> isinstance(item,str):
        text += <span class="hljs-string">" "</span> + item
    text.replace(<span class="hljs-string">"'"</span>, <span class="hljs-string">""</span>)
    text.replace(<span class="hljs-string">","</span>,<span class="hljs-string">""</span>)
    text.replace(<span class="hljs-string">'"'</span>,<span class="hljs-string">''</span>)


<span class="hljs-comment"># Create the wordcloud object</span>
wordcloud = WordCloud(width=<span class="hljs-number">800</span>, height=<span class="hljs-number">800</span>, background_color=<span class="hljs-string">"white"</span>)

<span class="hljs-comment"># Display the generated image:</span>
wordcloud.generate(text)
plt.figure(figsize=(<span class="hljs-number">8</span>,<span class="hljs-number">8</span>))
plt.imshow(wordcloud, interpolation=<span class="hljs-string">"bilinear"</span>)
plt.axis(<span class="hljs-string">"off"</span>)
plt.margins(x=<span class="hljs-number">0</span>, y=<span class="hljs-number">0</span>)
plt.title(<span class="hljs-string">"Most Used Subject Words"</span>, fontsize=<span class="hljs-number">20</span>,ha=<span class="hljs-string">"center"</span>, pad=<span class="hljs-number">20</span>)
plt.show()
</code></pre><p>Here's the output:</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1590540248193/cRq4Dar2T.png" alt="download.png" /></p>
<p>I created a histogram of the hours after midnight using seaborn.</p>
<pre><code><span class="hljs-keyword">import</span> seaborn <span class="hljs-keyword">as</span> sns
sns.distplot(emails[<span class="hljs-string">"SinceMid"</span>],bins=<span class="hljs-number">20</span>)
plt.title(<span class="hljs-string">"Hours since midnight"</span>)
</code></pre><p>Here is the histogram:</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1590540894969/IU6EtxMoC.png" alt="download_.png" /></p>
<p>You can check out <a target="_blank" href="https://python-graph-gallery.com/">python gallery</a> for more possible visualizations.</p>
<h2 id="conclusion">Conclusion</h2>
<p>I had fun writing this. I hope you did too while reading it. This goes without saying, and I encountered ERRORS while doing this [some of them I had never seen before]. When you get error messages, a good starting point is using the print statement to get insight and then googling the error message. 
Part II will also be published on this blog; it would focus on getting the body of the mail and not the subject as this one.</p>
<p>The full code can be found below:</p>
<div class="embed-wrapper"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a class="embed-card" href="https://github.com/yomaokobiah/email_analysis">https://github.com/yomaokobiah/email_analysis</a></div>
<p>Thank you for reading up to this point.</p>
<blockquote>
<p>Disclaimer: I encourage you to experiment outside what's written here. If you encounter bugs and you feel like getting me involved [after Googling], send me a DM on  <a target="_blank" href="https://twitter.com/yomaokobiah">Twitter</a>. I'd be happy to learn something new. Thank you in anticipation.</p>
</blockquote>
<p><strong>References/Credits</strong></p>
<ul>
<li><p><a target="_blank" href="https://developer.ibm.com/tutorials/pattern-analysis-of-emails-using-python/">An IBM tutorial on pattern email analysis using python 2</a> </p>
</li>
<li><p><a target="_blank" href="https://docs.python.org/3/library/imaplib.html#imaplib.IMAP4_SSL">IMAPLib Documentation</a> </p>
</li>
<li><p><a target="_blank" href="https://docs.python.org/3/library/email.html">Email package Documentation</a> </p>
</li>
</ul>
]]></content:encoded></item></channel></rss>