# Python Tutorial Komodo IDE only

## Overview

### Before You Start

This tutorial assumes:

- Python 2.3 or greater is installed on your system. ActivePython is a free distribution of the Python language. See the Debugging Python documentation for configuration instructions.
- You are interested in learning about Komodo functionality, including the debugger and the interactive shell.
- You are interested in Python and have some programming experience either in Python or another language.

### Python Tutorial Scenario

The Python Tutorial demonstrates how to use the Komodo debugger and interactive shell to explore a Python program. In particular, this tutorial examines a Python script that preprocesses files (similar to the C preprocessor). In this tutorial you will:

1. Open the Python Tutorial Project.
2. Analyze preprocess.py the Python program included in the Tutorial Project.
3. Analyze contenttype.py the Python module included in the Tutorial Project.
4. Run the program and generate program output.
5. Debug the program using the Komodo debugger.
6. Explore Python using the Komodo interactive shell.

See Interactive Shell and Debugging Programs for more information on this Komodo functionality.

## Opening the Tutorial Project

On the Start Page under Tutorials and Documentation, click Python Tutorial, or open the python_tutorial.kpf file from the samples/python_tutorials subdirectory of Komodo's user data directory

The tutorial project will open in the Places sidebar.

### Overview of the Tutorial Files

The following components are included in the python_tutorial.kpf project file:

- preprocess.py: The main program. This Python program parses input source files and produces output filtered on a set of rules and statements embedded in the original input source.
- preprocess current file: A run command for executing preprocess.py on the file currently open in Komodo.
- contenttype.py: A Python module used by the main program (preprocess.py) to identify the language of a given file.
- content.types: A support file used by the Python module contenttype.py.
- helloworld.html and helloworld.py: Sample files to process using preprocess.py.

### Open the Python Tutorial File

In the Places sidebar, double-click the preprocess.py file. This file opens in the Editor Pane; a tab at the top of the pane displays the filename.

## Analyzing the Python Files

This section reviews the code in preprocess.py and contenttype.py.

### Analyzing preprocess.py

In this step, you will analyze the Python program preprocess.py in sections. This program is an advanced Python script that is best addressed by focusing on certain areas within the code. Be sure that line numbers are enabled in Komodo (View|View Line Numbers) and that preprocess.py is displayed in the Komodo Editor.

About Preprocessors: A preprocessor is a program that examines a file for specific statements called "directive statements". These directive statements are interpreted, and the resulting program output is conditional based on those statements. In languages like C/C++, preprocessing is a common step applied to source files before compilation. The Python preprocess.py program mimics a C/C++ preprocessor using similar directive statements.

About Directive Statements: Preprocessor directive statements are dependent on the preprocessor program they are used within. In the preprocess.py program, a directive is preceded with a pound sign (#), and is located alone on a line of code. Placing a directive on a unique line ensures the statement is included in a file without breaking file syntax rules. Valid preprocess.py directives include:
        #define <var>[=<value>]
        #undef <var>
        #if <expr>
        #elif <expr>
        #else
        #endif
        #error <error string>

### Setting Up the preprocess.py Program

Komodo Tip: Notice that syntax elements are displayed in different colors. You can adjust the display options for language elements in the Preferences dialog box.

#### Lines 3 to 57 - Defining a Module Docstring

- help is defined in a module docstring
- docstrings are contained in triple-quoted strings (""")

Komodo Tip: See Explore Python with the Interactive Shell to examine these docstrings, and other Python elements, using the Komodo interactive shell.

Komodo Tip: Click on the minus symbol to the left of line 3. The entire section of nested help code is collapsed. This is called Code Folding.

#### Lines 59 to 65 - Importing Standard Python Modules

- Imports the following six modules:
- - `os`: operating system dependant helper routines
  - `sys`: functions for interacting with the Python interpreter
  - `getopt`: parses command line options
  - `types`: defines names for all type symbols in the standard Python interpreter
  - `re`: evaluates regular expressions
  - `pprint`: supports pretty-print output
  - `logging`: writes errors to a log file

#### Line 67 - Importing the contenttype Module

The custom `contenttype` module is used by the preprocess.py program and is not included in a standard Python installation.

- loads the `contenttype` module and imports the `getContentType` method

Komodo Tip: To interact directly with the `contenttype.py` module, see Explore Python with the Interactive Shell for more information.

### Defining an Exception Class

#### Lines 72 to 88 - Declaring an Exception

- `PreprocessError` class inherits from the Python `Exception` class
- an instance of the `PreprocessError` class is thrown by the `preprocess` module when an error occurs

Komodo Tip: Click the mouse pointer on the closing parenthesis ")" on line 72. Notice that its color changes to a bold red. The opening brace is displayed the same way. This is called "Brace Matching". Related features in Komodo are Jump to Matching Brace and Select to Matching Brace, available via the Code menu.

### Initializing Global Objects

#### Line 93 - Initializing log

- `log` is a global object used to log debug messages and error messages

Komodo Tip: On line 95, enter: `log = logging.` When you type the period, Komodo displays a list of the members in the `log` package. This is called AutoComplete. If the default key bindingscheme is in effect Pressing 'Ctrl'+'J' (Windows/Linux) or 'Cmd'+'J' (Mac OS X) also displays the AutoComplete list. Delete the contents of line 95.

#### Lines 98 to 111 - Mapping Language Comments

- `_commentGroups` is a mapping of file type (as returned by `content.types`) to opening and closing comments delimiters
- mapping is private to the `preprocess.py` module (`_commentGroups` is prefixed with an underscore to indicate that it is private to the `preprocess.py` module). This is a common technique used in variable, function, and class naming in Python coding).

Note that preprocessor directives recognized by the `preprocess.py` module are hidden in programming language-specific comments.

Komodo Tip: Use the Code sidebar, available in the Left Pane, to browse the general program structure of all currently open files. For each file, the code browser shows a tree of classes, functions, methods and imported modules. Python instance attributes are also displayed.

### Defining a Private Method

#### Lines 116 to 123 - Expression Evaluation

- `_evaluate method` is private to the `preprocess` module
- evaluates the given expression string with the given context

### Preprocessing a File

The `preprocess` method examines the directives in the sample source file and outputs the modified processed text.

#### Lines 129 to 140 - The preprocess Method Interface

The `preprocess` method takes three parameters as input:

- first parameter is the filename, `infile`
- second parameter specifies the output file (defaults to `stdout`); `outfile=sys.stdout`
- third parameter is an optional list of definitions for the preprocessor; `defines={}`

#### Lines 145 to 156 - Identifying the File Type

Examines how programming comments are delimited (started and ended) based on the type of file (for example, HTML, C++, Python).

- `getContentType` is called (imported earlier from the `contenttype.py` module) to determine the language type of the file
- file type is used to look up all comment delimiters (opening and closing language comment characters) in `_commentGroups`

#### Lines 158 to 166 - Defining Patterns for Recognized Directives

This section defines advanced regular expressions for finding preprocessor directives in the input file.

Komodo Tip: Use the Komodo Rx Toolkit to build, edit, or test regular expressions. New to regular expressions? The Regular Expressions Primer is a tutorial for those wanting to learn more about regex syntax.

#### Lines 178 to 303 - Scanning the File to Generate Output

This block of code implements a basic state machine. The input file is scanned line by line looking for preprocessor directives with the patterns defined above (`stmtRes`). This code determines whether each line should be skipped or written to the output file.

- source file is processed
- output is generated by a state machine implemented in Python

#### Lines 311 to 349 - Interpreting Command Line Arguments

The `main` method takes the text entered at the command line and uses the `getopt` module to parse the data into arguments. These arguments are then passed into the "preprocess" method.

- runs when preprocess.py is executed as a program rather than loaded as a module
- parses the filename and any defines (`-D`) set as command line arguments
- passes all data to the `preprocess` method

#### Lines 351 to 352 - Running the Main Method

- runs the `main` method when preprocess.py is executed as a program

### Analyzing contenttype.py

In this step, you will analyze the Python program contenttype.py in sections. This Python script is best addressed by focusing on certain areas within the code. Be sure that line numbers are enabled in Komodo (View|View Line Numbers) and that contenttype.py is displayed in the Komodo Editor Pane.

### Open contenttype.py

In the Places sidebar, double-click the contenttype.py file. This file opens in the Editor Pane; a tab at the top of the pane displays the filename.

### Setting Up the contenttype.py Module

The `contenttype.py` module is used by the main program, preprocess.py, to identify what programming language a particular file is written in based on the file extension and several other tests.

#### Lines 16 to 19 - Importing External Modules

- imports external modules used in this file (`re`, `os`, `sys`, `logging`)
- `logging` is not a standard module; it is new in Python 2.3

### Getting Data from content.types

#### Lines 29 to 31 - Finding the Helper File (content.types)

This section outlines the usage of the private `_getContentTypesFile` method located in the `contenttype` module.

- returns the complete path to the `content.types` file
- assumes the file is in the same directory as contenttype.py
- `_getContentTypesFile` is a private method that cannot be accessed from outside of the `contenttype` module

#### Lines 33 to 80 - Loading the Content Types from content.types

This section outlines the usage of the private `_getContentTypesRegistry` method located in the `contenttype` module.

- locates the `content.types` file and scans it to calculate three mappings to return, as follows:

    file suffix -> content type (i.e. ".cpp", a C++ implementation file)
    regex -> content type (i.e. ".*\.html?", an HTML file)
    filename -> content type (i.e. "Makefile", a Makefile)

- `_getContentTypesRegistry` is a private method that cannot be accessed from outside of the `contenttype` module.
- - Lines 44 to 45: gets the `content.types` file; if none is specified in the parameter for the method, `_getContentTypesFile` is called to find the system default
  - Lines 47 to 49: lists the three mappings to return (empty mappings are created here)
  - Lines 51 to 79: opens and processes the `content.types` file on a line-by-line basis
  - - scanning of the file stops when the last line is found, line 57
    - - Lines 58 to 78: each line is parsed to determine which of the three mappings it contains
      - an entry is made in the matching one
      - commented lines (starts with #) are ignored
  - Lines 79 to 80: closes the `content.types` file and returns the mappings

#### Lines 85 to 118 - Determining a File's Content Type

This section outlines the usage of the public `getContentType` method located in the `contenttype` module.

- takes one parameter (the name of the file to determine the content)
- returns a string specifying the content type (for example,
  `getContentType("my_web_page.htm")` returns "HTML" )
- `getContentType` is the only publicly accessible method in the module
- - Line 92: `_getContentTypesRegistry` is called to load the `content.types` file and to load the mappings
  - Lines 96 to 99: `filenameMap` is first checked to determine if the whole filename can be used to find a match
  - Lines 101 to 109: if the filename has a suffix (contains a '.'), the suffix map is then used to find a match
  - Lines 111 to 117: each regex in the regex map is then used to determine if it matches the filename
  - Line 118: returns the content type for the file (returns an empty string if no match was found by the above three mappings)

## Running the Program

This section reviews how to run the preprocess.py program using both a run command and the Komodo debugger.

### Using a Run Command

To start, generate simple output by running the program with the preprocess current file run command, included in the python_tutorial project toolbox.

1. Open the Source File: In the Projects sidebar, double-click the helloworld.html file. The file opens in the Editor Pane.
2. Run Command: In the Toolbox sidebar under the python_tutorial toolbox, double-click the preprocess current file command. A Preprocess Current File dialog box appears.
3. Preprocess Current File: In the Preprocessor Options text area, enter:
   `-D SAY_BYE`
   Click OK to run the program.
4. View Output: The helloworld.html file output is displayed on the Command Output tab as follows:
   `['path_to_file\\python_tutorials\\helloworld.html']
   <html>
   <head> <title>Hello World</title> </head>
   <body>
   <p>Hello, World!</p>
   </body>
   </html>
   `

Python Tutorial Tip: For more information about the `-D SAY_BYE` command, see Using the Debugger.

Komodo Tip: For more infomation on using run commands in Komodo, see the Run Command Tutorial.

### Using the Debugger

Generate output by running the program through the debugger without setting any breakpoints.

1. Run the debugger: Select the preprocess.py tab in the editor. From the menu, select Debug|Go/Continue. In the Debugging Options dialog box, click OK to accept the defaults.
2. View the Debug Output Tab: Notice the messages in the bottom left corner of the Komodo screen; these inform you of the status of the debugger. When the program has finished, program output is displayed in the Bottom Pane, on the right side. If necessary, click the Debug Output tab to display it.

Troubleshooting: "Why is this error message displayed?"
    preprocess: error: incorrect number of arguments:
    argv=['C:\\path_to_tutorial\\preprocess.py']
This error message is the expected output by the preprocess.py program when no source file or arguments are specified before it is run. The following instructions explain how to specify a file at the command line.

3. Specify a File to Process: In the Projects sidebar, double-click the file helloworld.html. Note the preprocessor directives inside the comments (#) in this file. Select the preprocess.py tab in the editor. From the menu select Debug|Go/Continue. In the Script Arguments text box on the Debugging Options dialog box, enter `helloworld.html`. Click OK.

Troubleshooting: "Why is this error message displayed?"
    <html>
    <head> <title>Hello World</title> </head>
    <body>
    preprocess: error: helloworld.html:5: #error: "SAY_BYE is not
    defined, use '-D' option"
This error message is the expected output by the preprocess.py program when no command-line arguments are specified with the source file helloworld.html. The following instructions explain how to specify a command-line argument with the source file to be processed.

4. Specify an Argument with the Source File: Select Debug|Go/Continue. In the Script Arguments text box in the Debugging Options dialog box, enter the following source file and argument: `-D SAY_BYE helloworld.html`. Click OK.

Troubleshooting: Specifying `-D SAY_BYE helloword.html` outputs the following:
    <html>
    <head> <title>Hello World</title> </head>
    <body>
    <p>Hello, World!</p>
    </body>
    </html>
In the helloworld.html file, if `SAY_BYE` is not defined, preprocessing generates an error. If `SAY_BYE` is defined, the preprocessor includes the line `<p>Hello, World!</p>` in the body of the output of the HTML. This demonstrates how a Python preprocessor can be used to conditionally include blocks of a source file being processed.

5. View the Debug Output Tab: Notice the HTML output and compare the result to the actual file helloworld.html.
6. View Rendered HTML: On the right side of the Bottom Pane, click the HTML tab. The rendered HTML for the helloworld.html file is displayed in the Bottom Pane. Click the Output tab to return to the HTML code.
7. Create New File: To create a new HTML file that will later contain the HTML code in the Bottom Pane, select File|New|File from Template. In the New File dialog box, select the HTML Template. Click Open.
8. Save the Output: Delete the contents of the new HTML file tab in the Editor Pane, and then select the contents of the Output tab on the Bottom Pane. Copy the contents to the new HTML file tab in the Editor Pane. Select File|Save As to save the file with a unique name.
9. Specify Another Source File: Go through steps 3 to 5 using the file helloworld.py in place of helloworld.html. Notice how the output displayed is now in Python, (for example, `print "Hello, World!"`). This demonstrates how the preprocess.py program can be used to process files written in different language types.

## Debugging the Program

In this step you will add breakpoints to the program and "debug" it. Adding breakpoints lets you run the program in sections, making it easier to watch variables and view the output as it is generated.

1. Set a breakpoint: On the preprocess.py tab, click on the gray margin immediately to the left of the code on line 347 of the program. This sets a breakpoint, indicated by a red circle.
2. Run the debugger: Select Debug|Go/Continue. In the Script Arguments text box on the Debugging Options dialog box, enter the following source file and argument (if not there from a recent run): `-D "SAY_BYE" helloworld.html`. Click OK.

Komodo Tip: Debugger commands can be accessed from the Debug menu, by shortcut keys, or from the Debug Toolbar. For a summary of debugger commands, see the Debugger Command List.

3. Watch the debug process: Notice that the line where the breakpoint is set (line 347) turns pink. Also, a yellow arrow appears on the breakpoint. This arrow indicates the position at which the debugger has halted.
4. View variables: On the Debug tab, click the Locals tab. If necessary, resize the pane by clicking and dragging the upper margin. On the Locals tab, notice the declared variables are assigned values. Examine the `infile` variable. This variable contains the name of the file specified above (helloworld.html).

Komodo Tip: What do the debugger commands do? Step In: Executes the current line of code and pauses at the following line. Step Over: Executes the current line of code. If the line of code calls a function or method, the function or method is executed in the background and the debugger pauses at the line that follows the original line. Step Out: When the debugger is within a function or method, Step Out executes the code without stepping through the code line-by-line. The debugger stops on the line of code following the function or method call in the calling program.

5. Step In: Select Debug|Step In until the debugger stops at line 129, the `preprocess` method. "Step In" is a debugger command that causes the debugger to enter a function called from the current line.
6. Set another breakpoint: Click on the gray margin immediately to the left of the code in line 145 to set another breakpoint. Line 145 is where `getContentType` is called.
7. Run the debugger: Select Debug|Go/Continue.
8. Step Over: When line 145 is processed, the variable `contentType` is assigned the source file's (helloworld.html) type (HTML). "Step Over" is a debugger command that executes the current line of code. If the line of code calls a function or method, the function or method is executed in the background and the debugger pauses at the line that follows the original line.
9. View variables: On the Debug tab, click the Locals tab. Examine the `contentType` variable. This variable contains the type of the source file; the type is "HTML" for helloworld.html.
10. Set another breakpoint: Click on the gray margin immediately to the left of the code in line 197 to set another breakpoint. Line 197 is inside of the loop where the source file helloworld.html is being processed.
11. Run the debugger: Select Debug|Go/Continue.
12. Add Watches for Variables: On the Debug tab, click the Watch tab. Click the New button in the lower-right corner of the Debug tab. An Add Variable dialog box appears. In the Add Variable dialog box, enter `lineNum` in the text box. Click OK. Notice that the `lineNum` variable and its value are displayed in the Watch tab. The `lineNum` variable is the line number of the line currently being processed in the source file helloworld.html. Follow the above steps again to enter a watch for the variable `line`. The `line` variable contains the actual text of the line currently being processed.
13. Run the debugger: Select Debug|Go/Continue. Notice how the variables in the Watch tab change every time the debugger stops at the breakpoint set at line 197. Also, notice the output in the right side of the Debug tab. This output changes as new lines are displayed.
14. Disable and Delete a breakpoint: Click on the red breakpoint at line 197. The red beakpoint is now white with a red outline. This breakpoint is now disabled. Click on the disabled white breakpoint. This removes the breakpoint, but does not stop the debugger.
15. Stop the Debugger: On the Debug menu, click Stop.

## Explore Python with the Interactive Shell

In this step you will use the interactive shell to explore the `contenttype` module. The Komodo interactive shell helps you test, debug, and examine your program. See Interactive Shell for more information.

If starting this section of the tutorial with currently open Python shells, please follow the steps below to ensure the Python shell's current directory is the Python Tutorial directory.

1. Close any Current Python Shells: Click the "X" button, located in the upper-right corner of the Shell tab, for each open Python shell.
2. Make sure python_tutorial is open (bold): If you haven't opened it before, click on the Python Tutorial link on the Start Page. If it's in your list of recent projects in the Projects pane, double-click it.

Start using the interactive shell with the Python Tutorial project files:

1. Start the Interactive Shell: On the Tools menu, select Interactive Shell|Start New Python Shell. A Python Shell tab is displayed in the Bottom Pane.
2. Import a Module: At the ">>>" Python prompt in the interactive shell, enter: `import contenttype`
   Notice that another ">>>" Python prompt appears after the import statment. This indicates that the `contenttype` module imported successfully.
3. Get Help for a Module: At the prompt, enter: `help (contenttype)`
   The help instructions embedded in the contenttype.py file are printed to the interactive shell screen. This is useful for easily accessing Python documentation without installing external help files.
4. Get Help for a Method in a Module: At the prompt, press the up arrow to redisplay previously entered commands. When `help (contenttype)` is redisplayed, enter `.getContentType` at the end of the command. The entire command is as follows:
   `help (contenttype.getContentType)`
   Press Enter. The help instructions for the `getContentType` method are printed to the shell screen. The ability to instantly access help on specific Python functions is a powerful use for the interactive shell.
5. Run a Method: At the prompt, enter: `contenttype.getContentType("helloworld.html")`
   Notice the output identifies the file type as `HTML`.
6. Run Another Method: At the prompt, enter: `contenttype.getContentType("helloworld.py")`
   Notice the output identifies the file type as `Python`.
7. Run a Final Method: At the prompt, enter: `contenttype.getContentType("test.txt")`
   Notice the output identifies the file type as `Text`. The `contenttype` module uses several tests to determine the data type used within a file. The test that determined that test.txt is a text file simply analyzed the file extension.

## More Python Resources

### Tutorials and Reference Sites

There are many Python tutorials and beginner Python sites on the Internet, including:

- The Python Home Page, the home of all that is Python.
- Python DevCenter, run by O'Reilly Networks, which provides access to tips, articles and other Python related items.
- ActivePython Documentation

### Preprocessor Reference

The preprocess.py program in this tutorial is a simplified version of another Python preprocess.py script available via http://starship.python.net/crew/tmick/#preprocess. The version available on starship.python.net is an advanced portable multi-language file preprocessor.