Dataphor Debugger

As of version 2.2 Dataphor features a D4 debugger, which allows for attaching, breakpoints, pausing, call stack traversal, and context stack inspection. In short it helps to diagnose issues in D4 code.

Edit

Architecture

Edit

Server Integration

The debugger is implemented as a class of which a session may have an instance. If a debugger instance is associated with the session, the session is acting as a debugger and it is said that the debugger is started relative to that session. A debugger may be attached to any number of other Sessions and Processes. If the debugger is attached to a Session, the debugger is automatically associated with all of that sessions processes, including any new processes which are started.

Edit

Debugger States

A started debugger may be in a paused or running state. The debugger may be paused either by explicit user request, or as a result of a process having encountered a breakpoint or thrown an exception. If a debugger is paused, all attached processes for that debugger are paused. While paused, it is possible to inspect the state of each attached process. And processes that start on a session that is attached to a paused debugger will immediately pause, and the converse is true of detached processes.

Edit

Locators

When a script is compiled, a debug Locator may be provided which provides debugger context for the given script. The locator consists of a document "locator name" as well as line and column numbers. The locator name is formatted as <type>:<data>. Current locator names are:

document:<library name>:<document name> - a Dataphor Frontend Server document.
file:<file name> - a File.
program:<program id> - a Script which was ran without locator information
operator:<operator name> - an Operator

Edit

Breakpoints

One or more breakpoints may be set by specifying a Locator. The set of Breakpoints is not persisted beyond the life of the debugger instance. If a running, attached process encounters a breakpoint, the debugger is paused. The process or processes which encountered a breakpoint leading to the pause is made available in the debugger state.

Edit

Process State

For each paused process, state is available to be read including:

The call stack, including the execution location at that each point.
The variable stack, with the present value of each variable. The variable name cannot currently be seen, nor can the value be set.

Edit

Step Over/Into

Step-over and Step-into allow the selected process to be advanced by a single statement. Step-into will step into any operator calls that are made, whereas step-over will not.

Edit

Break On Exception

Break On Exception is a mode on the debugger which, if set, will cause the debugger to pause if any of the attached processes throws an exception.

Edit

Programmatic Interface

The debugger logic and state is exposed in D4 as part of the Debug namespace in the System library. All actions, including starting the debugger, pausing, and inspection are performed through these D4 operators.

Edit

API

The D4 API for the debugger is as follows:

create operator Debug.GetDebuggers() 
	: table { Session_ID : Integer, BreakOnStart : Boolean, BreakOnException : Boolean, IsPaused : Boolean }
create operator Debug.SetBreakOnStart(ABreakOnStart : Boolean)
create operator Debug.SetBreakOnException(ABreakOnException : Boolean)
create operator Debug.GetSessions() : table { Session_ID : Integer }
create operator Debug.GetProcesses() : table { Process_ID : Integer, Locator : String, Line : Integer, LinePos : Integer, DidBreak : Boolean }
create operator Debug.AttachProcess(AProcessID : Integer) 
create operator Debug.DetachProcess(AProcessID : Integer) 
create operator Debug.AttachSession(ASessionID : Integer) 
create operator Debug.DetachSession(ASessionID : Integer) 
create operator Debug.GetCallStack(AProcessID : Integer) 
	: table { Index : Integer, Description : String, Locator : String, Line : Integer, LinePos : Integer, Location : String, Statement : String }
create operator Debug.GetStack(AProcessID : Integer, AWindowIndex : Integer) 
	: table { Index : Integer, Name : Name, Type : String, Value : String }
create operator Debug.GetSource(ALocator : String) : String
create operator Debug.GetBreakpoints() 
	: table { Locator : String, Line : Integer, LinePos : Integer }
create operator Debug.ToggleBreakpoint(ALocator : String, ALine : Integer, ALinePos : Integer) : Boolean
create operator Debug.Start() 
create operator Debug.Stop() 
create operator Debug.WaitForPause() 
create operator Debug.Pause() 
create operator Debug.Run() 
create operator Debug.StepOver(AProcessID : Integer) 
create operator Debug.StepInto(AProcessID : Integer)

Edit

User Interface

The Dataphoria environment provides a complete visual user interface for the above API. Using a combination of D4 commands and the user interface is possible, but this is not recommended because the state of the UI may become out of sync resulting in surprising behavior. For example, if the debugger were stopped using a D4 command, the UI would not be informed of this and a subsequent debugging action may fail with an error.

Dataphoria Debugger

Edit

Attaching

The debugging process starts by attaching to a session or process. This is done by opening up the Process or Session list and selecting Attach. Double clicking in the process or session list is a useful short-hand for Attach. The debugger cannot be attached to its own session, so using the debugger to debug execution happening in Dataphoria requires launching another Dataphoria instance pointing to the same Dataphor server. CAVEAT: A Dataphoria instance can debug another Dataphoria instance which is hosting the Dataphor server in-process, but will not be able to debug anything coming from the main (UI) thread. This is due to a seemingly arbitrary restriction imposed by the .NET framework on the SignalAndWait synchronization primitive. To work around this, do the debugging from the Dataphoria which is hosting the Dataphor server in-process, and do the testing/running from a Dataphoria which is connected out-of-process.

Edit

Breakpoints

Either before or after the debugger is attached, one or more breakpoints may be set either in an open document, or by selecting ‘’Open’’ on an operator in the Dataphor Explorer tree. A breakpoint is toggled by pressing F9 or clicking in the icon bar to the immediate left of the editor text. A breakpoint is retained even if the editor windows is closed. As was mentioned previously, breakpoints are retained only for the duration of a debugger session; once the debugger is stopped, the breakpoints are lost.

Edit

Debugged Processes

The Debugged Processes list shows the current processes that the debugger is attached to and which of those processes is the currently ‘’selected’’ one. This list is similar in nature to the thread list in common debuggers. The selected process is used to determine which call stack is displayed and thus indirectly which the current location (discussed next). A process may be made the selected one by pressing the Select button from the toolbar above the grid, or by double-clicking on the row in the grid.

Edit

Call Stack

The Call Stack list displays the series of operator calls leading to the current location of the selected process. Like the debugged processes list, one call stack index may be ‘’selected’’, which determines the current line displayed in the text editor as well as the context in which variables are shown in the Stack list. A call stack item may be selected by pressing the Select button from the toolbar above the grid, or by double-clicking on the row in the grid.

Edit

Stack

The Stack list provides the list of local variable and arguments for the currently selected Call Stack index of the currently selected process. At present, only the index and value are displayed, not the variable name. This is because at runtime we no longer have the compiler symbols (identifier names). We plan to rectify this in the future.

Edit

Instructions

To get started using the debugger:

Open the Client or Dataphoria and prepare to execute the sequence of events you wish to diagnose. This is the session you will be debugging and it must not be the host of an in-process Dataphor server.
From an instance of Dataphoria connected to the same Dataphor server, open View Sessions under the Debug menu. This Dataphoria instance may be a Dataphor server in-process host, but must not be the same instance as the one you wish to debug.
Select the session which represents the debug target and double-click or press the Attach button.

From here, the steps depend on what kind of issue is being diagnosed:

Process is freezing – to diagnose this, run the process until it enters the frozen state, the select Pause from the Debug menu. The selected debug process, call stack, and source code in the text editor should locate to the statement currently being executed. From here, execute statements line-by-line using step-over and step-into to determine why the logic is not completing. Note: if the freezing is not a result of D4 logic but is resulting from a long running query from a storage engine, then the pause will also not complete until the long-running query completes.
Logic not behaving as expected – to diagnose such issues, navigate to one or more locations in the source code where you suspect an unexpected decision is being made and place breakpoints. To navigate to the source location, either open up the document or navigate to the Operator in the Schema section of the Dataphor Explorer tree and select Open. To set a breakpoint, press F9 or left click in the icon tray to the immediate left of the text editor. A red dot should appear and the line should be highlighted in red. When and if the execution of the debugged session reaches a breakpoint, the debugger should pause and navigate the current process, call stack index, and source code location to the breakpoint. At this point, use the View Stack command in the Debug menu to determine the current state of local variables and arguments to try to ascertain the problem. If necessary, use the step-over and step-into commands to advance execution one statement at a time.
Determining why an error is being raised – to diagnose an exception, turn on Break On Exception in the Debug menu. When set, any exception raised by the debugged processes with cause the debugger to pause. Once paused, use the View Stack command to inspect local variables or set breakpoints earlier in the code to determine what lead to the exception.

Edit

TO DO

There are many enhancements possible for the debugger, including:

Load or preserve the local variable names
Persistance of breakpoints
List the breakpoints in Dataphoria
Step Into Specific
Run To
Detection of breakable lines
Improve line-number and offset information in the Parser/Compiler.
Ability to set the value of local variables
Improve handling of resumption from break on exception (re-stops for internal re-throws)
Conditional breakpoints
Inter-expression debugging
Add Break operator to manually cause a break.