Search Wikipedia from Command Prompt

This tutorial shows how to quickly setup python scripts to search Wikipedia  right form the terminal.  This would be ideal for people who want to quickly read up on a topic without having to open a browser , wait for it to load and search. Terminal would do the job more quickly and efficiently.

The way to solve this is to use the Wikipedia API  to send a http request to the website as a query action in json format and get the response is back as a json Object. This could be implemented using the request  module in python.

The next step is to parse this response json object. I found that Beautiful Soup can be used to do that. This was one of the best options available . Once the parsing was complete, we only have to display the data.

I found 2 scripts to do just that. These scripts however don’t use Requests module but use urllib and urllib2.

Advantage :
The advantage of using this is that, only a brief summary of the topic under search will be showed. And most of the time it is the only thing we want.

The second thing is that if there are sections in the topic, it will be shown.

Disadvantage :
What this lacks is that, it does not have a good pattern matching for searches, that is if it doesn’t ind the exact words in the article it will not be returned. This also happens if each result has multiple result.
Sometimes the data returned is either very less or a lot.

Procedure:

First download and save these 2 python file:
wikipedia.py  : This python Program  is used to form the url for the search term to get the article from wikipedia.com. Once the URL is formed, we send a request using the urllib python library.We perform the search and get the whole data from Wikipedia.

wiki2plain.py : This program is used to convert the full   document received  from previous program to readable text format. Usually, the response from the previous program is in the form of json/html. Thus we use this program to parse the json/html and get meaningful data on the topic.

Then create a python file and name it wiki.py and paste the following script in it:


from wikipedia import *
from wiki2plain import *

lang = ‘simple’
wiki = Wikipedia(lang)
try:
    data1 = raw_input(“enter searh query: “)
    raw = wiki.article(data1)
except:
    raw = None

if raw:
    wiki2plain = Wiki2Plain(raw)
    content = wiki2plain.text
    print content
else:
    print “No text returned”

This code just calls the previously downloaded files and allows you to dynamically enter a topic and search Wikipedia.
Save it in the same folder as the other two files and run  wiki.py

Enter the search term and get the results.

Screenshots :

Screenshot from 2014-04-20 22:21:28

Screenshot from 2014-04-20 22:21:06

Installing Network Simulator 2 (NS2) on Ubuntu 14.04

1 Introduction

Network simulators are tools used to simulate discrete events in a network and which helps to predict the behaviours of a computer network. Generally the simulated networks have entities like links, switches, hubs, applications, etc. Once the simulation model is complete, it is executed to analyse the performance. Administrators can then customize the simulator to suit their needs. Network simulators typically come with support for the most popular protocols and networks in use today, such as WLAN,UDP,TCP,IP, WAN, etc.

Most simulators that are available today are based on a GUI application like the NCTUNS while some others incl. NS2 are CLI based. Simulating the network involves configuring the state elements like links, switches, hubs, terminals, etc. and also the events like packet drop rate, delivery status and so on. The most important output of the simulations are the trace files. Trace files log every packet, every event that occurred in the simulation and are used for analysis. Network simulators can also provide other tools to facilitate visual analysis of trends and potential trouble spots. Most of the simulation is performed in discrete time intervals where events that are in the queue are processed one after the other in an order.

Since simulation is a complex task, we cannot guarantee that all the simulators can provide exact or accurate results for all the different type of information. Examples of network simulators are: ns, NCTUNS, NetSim, etc.

ns2 is a name for series of discrete event network simulators like ns-1, ns-2 and ns-3. All of them are discrete-event network simulators, primarily used in research and teaching. ns2 is free software, publicly available under the GNU GPLv2 license for research, development, and use.

This post deals with the installation of “ns2” also called the “network simulator 2” in Ubuntu 14.04.

2 Download and Extract ns2

Download the all in one package for ns2 from here

The package downloaded will be named “ns-allinone-2.35.tar.gz”. Copy it to the home folder. Then in a terminal use the following two commands to extract the contents of the package.:

cd ~/
tar -xvzf ns-allinone-2.35.tar.gz

All the files will be extracted into a folder called “ns-allinone-2.35”.

3 Building the dependencies

Ns2 requires a few packages to be pre installed. It also requires the GCC- version 4.3 to work correctly. So install all of them by using the following command:

sudo apt-get install build-essential autoconf automake libxmu-dev

One of the dependencies mentioned is the compiler GCC-4.3, which is no longer available, and thus we have to install GCC-4.4 version. The version 4.4 is the oldest we can get. To do that, use the follwoing command:

sudo apt-get install gcc-4.4

The image below shows the output of executing both the above commands. If you have all the dependencies pre-installed, as I did, the output will look like the image below:

Once the installation is over , we have to make a change in the “ls.h” file. Use the following steps to make the changes:

Navigate to the folder “linkstate”, use the following command. Here it is assumed that the ns folder extracted is in the home folder of your system.

cd ~/ns-allinone-2.35/ns-2.35/linkstate

Now open the file named “ls.h” and scroll to the 137th line. In that change the word “error” to “this->error”. The image below shows the line 137 (highlighted in the image below) after making the changes to the ls.h file.To open the file use the following command:

gedit ls.h

Save that file and close it.

Now there is one more step that has to be done. We have to tell the ns which version of GCC will be used. To do so, go to your ns folder and type the following command:

Sudo gedit ns-allinone-2.34/otcl-1.13/Makefile.in

In the file, change Change CC= @CC@ to CC=gcc-4.4, as shown in the image below.

 

For installation, usage and  more, read my full story here.

How to install and use Waartaa IRC client as a local standalone application

1 Introduction

Waartaa is an open source communication and collaboration tool. It is an IRC client focussed to overcome the subtle disadvantages of existing clients. It is a web based IRC client as a service, facilitating some extremely useful services such as: centralised logging, unique Identity, global access, notifications, responsiveness to devices and displays across multiple clients.

This project is opensourced under the MIT Licence and built on the Meteor Platform and written using Meteor JS. The database at the backend is MongoDB and it has also incorporated node-irc, an IRC client library written in JavaScript for Node.

2 Installing dependencies

Waartaa requires that a few dependencies be satisfied before it can be installed. The main dependencies are Node JS and curl. It involves adding the Node JS repositories, updating the list and then installing curl. Use the following commands, one after the other.

sudo add-apt-repository ppa:chris-lea/node.js
sudo apt-get updatesudo apt-get install nodejssudo apt-get install curl

3 Downloading Waartaa

Waartaa can be downloaded either through git or by direct download. Either of the method gives the same packages.

3.1 Download through git

Use the following command to download and extract Waartaa through git.

git clone –recursive https://github.com/waartaa/waartaa.git

3.2 Direct download

You can download the zipped folder here. Once downloaded, extract it into your home folder.

4 Setup and starting the application

Before Waartaa can be run, we have to set it up with the server details and then install it onto our system. Use the following commands:

cd waartaa./setup.sh

Now wait for the setup to finish.

Once the setup is complete, we now have to configure the server hosts file. Open the file settings-local.js file by using the following commands:

cd appgedit server/settings-local.js

Once the file is opened, scroll to the very bottom and add your details such as the “SUPER_USER”, “SUPER_USER_EMAIL”, “SUPER_USER_PASSWORD” and “SECRET_KEY”. An example is given in the image below. Do remember to change it to your preferred credentials.

This completes the setup. We can now run Waartaa. The first time you run it, it will download the meteor platform tools and the dependencies it needs to run the application like the MongoDB packages. But the 2nd time onwards, it runs immediately. To run the application use the following command:

meteor

Make sure that while running the meteor command, you are always present in the app folder of Waartaa, which is the meteor’s project folder. Once the command successfully launches the application, your terminal will look similar to the image below:

If upon running the command “meteor” you get an error saying: ” project uses Meteor 1.0.2.1, which is not installed and could not be downloaded. Please check to make sure that you are online.”, then all you have to do is run the “meteor” command again and it will download the required version and start the application.

For using the GUI  and oter features, read my full article here.

Installation and Usage Guide for CSF firewall

1 Introduction

CSF stands for Configserver security and firewall. CSF is a configuration script built to provide better security to servers , at the same time providing a large number of configuration options and features to configure and secure with extra checks to ensure smooth operation. It helps in locking down public access and to restrict what can be accessed like only e-mails or only websites, etc. To add more power to this, it comes with a Login Failure Daemon (LFD) script that runs all the time to scan for failed attempts to login to the server to detect bruteforce-attacks. There are an array of extensive checks that lfd can perform to help alert the server administrator of changes to the server, potential problems and possible compromises.

  • Login Tracking
  • Process Tracking
  • Directory Watching
  • Advanced Allow/Deny features
  • Block Reporting
  • Port Flood Protection

And many more. This post does not cover all the features and thus for more detailed information about each of the features read the “readme.txt” file from the csf folder that we will download.

2 Downloading and Installing

The first step involves removing any previous version of csf that might have been downloaded and then downloading the latest version. To perform these use the following two commands:

rm -fv csf.tgz
wget http://www.configserver.com/free/csf.tgz

Now we extract the tar file in the home directory and move into the csf directory.

tar -xzf csf.tgz
cd csf

The steps till here are shown in the image below.

Now we are ready to install, but before we can, we will need to have root privileges else we will not be able to install. So, use the following command to gain root privilege and type in the password if asked.

sudo su

Install CSF using the following command:

sh install.sh

Once the installation is over successfully, the output will look similar to the image below.

Once the installation is complete, we can do the verification.To do so, we test if our system has all the required iptables modules. Now when this is run, it might indicate that you might not be able to run all the features but that is alright. This test can be considered as PASS as long as the script doesn’t report any FATAL errors. To test it, use the following command:

perl /usr/local/csf/bin/csftest.pl

My result of running this test is shown in the image below:

3 Remove other firewalls

It is important to remove older firewalls or any other firewalls setup to protect the server. This is because the conflict of Firewalls can lead to failures or inaccessibility. You should also not install any other iptables firewall and if it already exists, then it has to be removed at this stage. Most of the systems is likely to have APF+BFD firewalls and has to be removed. So use the following command to detect and remove them if they exist.

sh /usr/local/csf/bin/remove_apf_bfd.sh

I didn’t have it pre-installed , so the output of the command in my system looked like the image below:

Read my full post on HowToForge here.

Startup Applications: automatic execution of the most used programs during every user login

During every login or bootup, the OS performs a number of tasks in the background to get the OS up and running and ready to be used. Customizations in Linux OS like Ubuntu can be done very easily, that allows an user to add any number of “custom” tasks or programs to the system to be performed every time the user logs in or boots up the OS.

We can choose a certain number of tasks to be automated, thereby reducing the repetitive work of the user to quite an extent. The tasks that can be automated include running a certain program like a custom system check that the user might want to view, or to run applications like the web browser with certain predefined websites. Or it can be automating system control programs that needs to be run during each bootup or even login. Ubuntu allows us to do these kind of automation in a very simple manner, by providing us a GUI based software called the “Startup Applications”.

Though this may provide a lot of convenience, it is important to remember not to overuse this feature. Users might face problems when a large number of tasks are provided or selected to be automated. The main problem this leads to is slowing down your system at startup. Hence, it is very important for the user to NOT over utilise this functionality and only automate the most required or used tasks.

So let’s begin. First thing to do is to open the Startup Applications. To do so, in the desktop Dash, type:

“startup applications” as shown in the image below.

Ubuntu Desktop - Startup applications

Click on the “Startup Applications” icon, and the Startup Applications window opens. It will look similar to the image below. Note that you might have a different set of entries in your window than the ones that can be seen in the image below.

Startup applications preferences

Once this is open, we can start with the automation of tasks. This posts will demonstrate some of the tasks, but many other tasks that are important to you can be added the same way. So here are a few examples:

1 Starting a browser with a predefined web page

Every build of Ubuntu comes with Firefox inbuilt. We can use that to open websites that we check regularly. For example, I like to read Linux news, from the website “lxer.com”. To open this website every time, click on the add button present on the upper right side of the Startup Applications window.

Another window opens. In the new window, type the following in the command field:

Firefox “http://www.lxer.com”

Fill out the name as well and use the comment field to describe your new task. An example is shown in the image below. Once that is done, click on the “Add” button and you will see an entry named “lexr Linux news” (in this case) in the Startup Applications window.

Add startup program

This way, a browser can be opened automatically with any website or search query. For example, if the first thing you do is view the trade stock market, then you can click “Add” and enter the command to go directly to a trade website. Example:

Firefox “NASDAQ”

2 Running system management functions automatically

Certain system management functions or programs require to be started each time a user logs in. Consider the situation of system overheating experienced by many Ubuntu users. In this case, most of the users install a software called “TLP”. And in many cases, TLP doesn’t always start on its own automatically. In such a case, rather than to run it manually each time, the Startup Applications can be used to run it automatically, by following the similar steps. So, click on the “Add” button and use the following command in the command field:

sudo tlp start

For the name and the comment field, refer to the image below:

sudo tlp start

Unlike the first example, this example of TLP , the program is started in the background and not visible to the user. Thus in this way, a user can start a variety of background processes automatically that relate to system management functions.

To read my full post head over to HowToForge: here

First Update for Ubuntu Dual Boot Installer from Canonical

On 7th jan 2014 the first update to the Ubuntu dual boot installer for touch devices was released. The mail sent by the Canoical team says that this is the first update since Ubuntu and Android dual boot developer preview was announced in Dec 23rd 2013.  Ubuntu Dual Boot Installer is provided as a tech preview for developers who want to run Ubuntu and Android on a single device.Canonical also  has still kept the warning that it is not for regular users as its not a fully stable version. You have to remember that the Ubuntu Dual Boot Installer is still in development and you might experience problems.

Canonical-Releases-First-Update-for-Ubuntu-Dual-Boot-Installer

Here is a complete download and tutorial from the ubuntu wiki : https://wiki.ubuntu.com/Touch/DualBootInstallation

This version however was extensively tested on the Nexux 4 and yet to be tested on Nexux 10. According to the changelog, a warning for unsupported devices is now shown, Ubuntudualboot app is now installed in Ubuntu, and the Install/Download progress bar now fits the screen size in Nexus 10.
Also, Ubuntuinstaller now supports resume for downloading images, and the Ubuntu Installer Application now displays channel chooser on CM 10.2.

this release comes with  6 major bug fixes that can be found here.
And the list of known bugs is listed over here.

 

Interesting commands on Linux :SET 1

Here are some fun commands you can try on your Linux system probably when you are bored or want to try something new and something different. I really liked these commands and hope that you will enjoy using it too.

  1. cowsay
    you can make an image of cow appear on the terminal and  show any text that you want , as if the cow was saying it –
    So install cowsay using one of the following command :
    sudo apt-get install cowsay                     (for Debian based OS)
    yum install cowsay                                   (for Red Hat based OS)
    here is how the uoyput will look:
    cowsay1If you want to have a coloured cow to show any text install xcowsay .  Install it same way as cowsay but replace cowsay with xcowsay.
    Run it and ul get a coloured cow some where on your screen.
  2. Steam Locomotive (sl)
    this command “sl” runs a train across your terminal. Its really cool!!
    So install it using one of the commands:
    sudo apt-get install sl                            (In Debian like OS)
    yum -y install sl                                    (In Red Hat like OS)
    run the command  “sl” in he terminal, you should see something like this:
    sl2
  3. toilet
    This command itself is funny.. but what it does is takes a text string and builds the text in some different forms like using symbols some colours,etc
    To install use one of the command:
    sudo apt-get install toilet
    yum install
    toilet

    run the commands as :
    toilet seeTheSource
    toilet3or run the command: toilet -f mono12 -F metal seeTheSource
    toilet4
  4. telnet
    telnet is a network protocol and it causes stories to be run like some kind of animations. Nothing has to be installed here, all you need is a working internet condition.use the command:
    telnet towel.blinkenlights.nl
  5. cmatrix
    this command is used to create the matrix effect on a terminal.install it using the suitable command:

    sudo apt-get install cmatrix
    yum install cmatrix
    
    cmatrix
  6. espeak
    this command is used for text to speech. So you can configure it to be a welcome sound when your OS loads. Or use it to read text just like that.
    install espeak using one of:

     sudo apt-get install espeak 
     yum install espeakthen run it as: espeak "this is see the source" 
     or any other text you want. And have fun.

So all these commands are really fun to use, there are lots more available.. I will post about more useful commands in the coming posts!!

Animated desktop: For ubuntu 13.04 Raring /ubuntu 13.10 Saucy

An animated desktop looks amazing, and what can be more cooler than having the ability to setup an animated desktop with a video of your own choice??
This post will give the detailed steps to setup an animated desktop on Ubuntu 13.04 and 13.10 . The procedure works completely fine with older versions as well (but the older versions have an easier method). So to get the background live with a video of your choice , a few requirements have to be met. Few software have to be downloaded and installed. And  for videos which you have chosen to work , they must be in the .wmv video format.

Check out the video below to see how the animated desktop looks like. And follow the procedure below to make your desktop look like that. The link to the animations shown in the video is given at the end of this post. But remember this video is for Ubuntu 12.04 so the procedure for 13.04 and 13.10 is not this easy!!!

procedure for Ubuntu 13.04/13.10:–

step 1:
install the following software to support and play your video as a background. Copy paste these following lines one by one in the terminal :

  • sudo apt-get install mplayer
  • sudo apt-get install rar
  • sudo apt-get install unrar
  • sudo apt-get install ubuntu-restricted-extras
  • sudo apt-get install zenity

so this will install all the prerequisites that were needed. Since Zenity has some error, the installation cannot be done correctly on Ubuntu 13.04 and 13.10 which is why a different procedure has to be followed.

step 2:
Here we use the a-desk script for installation. the a-desk is a shell script that automates all the work that has to be done to set the desktop. So go ahead and download the a-desk zip folder from HERE .
Once downloaded extract it to get a folder named “ad” and place it where you want it to be. Now open the terminal and go to the location where you have stored the “ad” folder.
For example. if you have put he folder under /Home then use the command :
cd /home/ad

Once you are in the folder, run the shell script a-desk.installer using the following command:

sudo sh a-desk.installer

This will run the script and give you the options to select your language and the version. That is you can choose to get the 32bit version and 64 bit version .Once you select it it will start installing and once it is done  The “OK” button on the installer window will be highlighted. Click on it to finish the installation.

step 3:
Now go to the terminal and type the command :

~/.gnome2/nautilus-scripts/a-desk
This will open a window and you have to select the option atomic city. This will load the video atomic city to the background. Atomic city is a video that comes by default with he a-desk script.So thats it.. now you will have an animated desktop.

In my next post I will give the procedure to use customised videos as your animation for the desktop and also to change the animation to the videos present in the link given below!!

More amazing videos to setup from the background can be found from : HERE
Go to that website and download any video that you like and make it your animated desktop.

Disclaimer: When the animated desktop is setup , it doesn’t stay after the bootup. That is , this procedure doesn’t work on startup and you would have to repeat the procedure all over again to set it up. This is something I am working on and as soon as I find a solution, I will post it.

Things to do after installing ubuntu

Ubuntu 12.04 is the latest LTS (long term support)  OS available.But after it is installed there are few things that can  be done – to make it more user friendly , to improve its performance and also to make it look  better.As i started using ubuntu, I started customizing and tweaking it . And while trying out new things like changing the startup sound, creating a 3d desktop ,etc  I found that I could have an operating system that  is so unique that, it represents me. A freedom which no one  will  find  in  any of the proprietary OS. Though there are tons of customization  that can be done,here are a few things to start with..

Keeping the OS updated
Once Ubuntu is installed it is important to update the repositories so that all the features are updated and bugs if any are fixed.It can be done in two ways. One way is to use the software updater.To use it type  “software” in the dash and click on  the “software updater” application. This will automatically check for the updates and asks you weather to install it or not.Select install and let it do the process.  

 

software

the other way is to use the terminal.So type the following command in the terminal:
                        sudo apt-get update && sudo apt-get upgrade

Once this is done, now the other software applications can be installed.Again  these software can be installed in two ways.
The first is by using the “Ubuntu software center”. Here you can search and install the software directly. The screenshots below shows installing Blender
animation software.
The second method is to use the command line.You can google  any particular software that you want to install and get the commands to install them.

software center

softwarecenter2blender

some of the  Software to install-
This list mentions some of the software that are needed to perform some basic tasks:

  1. VLC madia player – this is the best open source video and audio player known, having  the ability to play any kind of media format which makes it a                                                     preferred choice by most of the users.
  2. Gimp – This is an image editor and has the capability to perform almost all the operations photoshop can.
  3. Chromium – This is an open source browser and is a substitute to the “google chrome” browser.
  4. Jupiter – This is an optimization software and it helps increasing the battery backup and also reduce the heating of the laptop.
  5. Synaptiks –This package helps configure gestures for the laptop’s touchpad.
  6. Gparted – Gparted partition editor  can be used to to maintain and edit the disk drives of the system.
  7. Adobe flash plugin – This plugin is mainly  necessary , while video streaming on the web.The browser asks you to install the plugin when you try watching                                     a youtube video on the newly installed OS.
  8. Bumblebee  –  Ubuntu by default doesn’t  use the external graphics card. But   bumblebee can be  to run specific software using the  external GPU.

All the above mentioned software other than bumblebee can be downloaded through the “software center”. These are just a few to start with.
Hundreds of other software, tweaks and customization can be done to make your ubuntu awesome .The only way to use ubuntu is by experimenting and trying out new things on it , so  that it can make your system  efficient and productive for you to use.