277 lines
16 KiB
TeX
277 lines
16 KiB
TeX
\documentclass[journal,twoside]{IEEEtran}
|
||
|
||
\usepackage{amsmath}
|
||
\usepackage{amsfonts}
|
||
\usepackage{graphicx}
|
||
\usepackage{hyperref}
|
||
\usepackage{cite}
|
||
\usepackage{multirow}
|
||
\usepackage{caption}
|
||
|
||
\captionsetup[table]{skip=10pt} % Adjust the value of skip as needed
|
||
% Custom title setup
|
||
\newcommand{\hwtitle}[1]{
|
||
\section*{\centering #1}
|
||
}
|
||
|
||
\begin{document}
|
||
|
||
% Title and author
|
||
\title{Experiment Results}
|
||
\author{Ali Can Zeybek, Furkan Samet Akıncı}
|
||
\markboth{Experiment Results}{Ali Can Zeybek, Furkan Samet Akıncı}
|
||
|
||
\maketitle
|
||
|
||
% Introduction Section
|
||
\section{Introduction}
|
||
In this report we implemented and tested two given programs. Decision table testing, equivalence testing and boundary testing done for each and compared a) different testing techniques for same application b) same testing technique for different programs.
|
||
|
||
% Experiment 1 Section
|
||
\section{Experiment 1}
|
||
In this program there are 3 input variables as `number\_of\_items` , `total\_cost` and `delivery\_time`. First two variables are numberic while the third parameter is an enum type.
|
||
\subsection{Decision Table Testing}
|
||
Decision tree testing for program 1 generated 12 different test cases, it is expected since for inputs `number\_of\_items` and `total\_cost` generate two possible paths of execution,
|
||
|
||
\bigskip
|
||
\resizebox{\columnwidth}{!}{$
|
||
\text{Branching due to }(\#items) =
|
||
\begin{cases}
|
||
\text{True}, & \text{if } number\_of\_items \leq 30, \\
|
||
\text{False}, & \text{if } number\_of\_items > 30.
|
||
\end{cases}
|
||
$}
|
||
\bigskip
|
||
|
||
\resizebox{\columnwidth}{!}{$
|
||
\text{Branching due to }(total\_cost) =
|
||
\begin{cases}
|
||
\text{True}, & \text{if } number\_of\_items \leq 100, \\
|
||
\text{False}, & \text{if } number\_of\_items > 100.
|
||
\end{cases}
|
||
$}
|
||
\bigskip
|
||
|
||
while `delivery\_time` can generate 3 values as `NEXT\_DAY`, `SECOND\_DAY` and `THIS\_WEEK`. Hence,
|
||
2 x 2 x 3 = 12.
|
||
|
||
|
||
\begin{table}[ht]
|
||
\centering
|
||
\resizebox{\columnwidth}{!}{%
|
||
\begin{tabular}{|c|c|c|c|}
|
||
\hline
|
||
\textbf{number\_of\_items} & \textbf{total\_cost} & \textbf{delivery\_time} & \textbf{Expected Outcome} \\ \hline
|
||
$\leq 3$ & $\leq 100$ & NEXT\_DAY & 25 \\ \hline
|
||
$\leq 3$ & $\leq 100$ & SECOND\_DAY & 10 \\ \hline
|
||
$\leq 3$ & $\leq 100$ & THIS\_WEEK & number\_of\_items * 1.50 \\ \hline
|
||
$\leq 3$ & $> 100$ & NEXT\_DAY & 35.00 \\ \hline
|
||
$\leq 3$ & $> 100$ & SECOND\_DAY & 15.00 \\ \hline
|
||
$\leq 3$ & $> 100$ & THIS\_WEEK & 10.00 \\ \hline
|
||
$> 3$ & $\leq 100$ & NEXT\_DAY & number\_of\_items * 6.00 \\ \hline
|
||
$> 3$ & $\leq 100$ & SECOND\_DAY & number\_of\_items * 2.50 \\ \hline
|
||
$> 3$ & $\leq 100$ & THIS\_WEEK & 0.00 \\ \hline
|
||
$> 3$ & $> 100$ & NEXT\_DAY & number\_of\_items * 7.50 \\ \hline
|
||
$> 3$ & $> 100$ & SECOND\_DAY & number\_of\_items * 3.50 \\ \hline
|
||
$> 3$ & $> 100$ & THIS\_WEEK & number\_of\_items * 2.50 \\ \hline
|
||
\end{tabular}}
|
||
\caption{Test cases generated for decision table testing.}
|
||
\label{tab:delivery_cost}
|
||
\end{table}
|
||
|
||
The total test case generation and implementation took 55 minutes. While running 12 test cases took 0.008 seconds \footnote{Test execution times are reported by mvn}, running the full `mvn test` command took 4.59 seconds \footnote{`time mvn test` command used for timekeeping} \footnote{Tested setup has 32G ram Ryzen 7 5800X running Debian 12 and openjdk 17.0.13 }
|
||
|
||
\subsection{Equivalence Testing}
|
||
|
||
For equivalance testing valid and invalid partitions of input variable domains are selected.
|
||
|
||
\subsubsection{Equivalence Partitions}
|
||
|
||
Following are the valid and invalid partitions of program 1.
|
||
|
||
\begin{table}[ht]
|
||
\centering
|
||
\resizebox{\columnwidth}{!}{%
|
||
\begin{tabular}{|c|c|c|c|}
|
||
\hline
|
||
\textbf{Input} & \textbf{Partition Type} & \textbf{Partition Condition} & \textbf{Selected Value} \\ \hline
|
||
number\_of\_items & Valid & $\leq 3$ & 3 \\ \hline
|
||
& Valid & $> 3$ & 4 \\ \hline
|
||
& Invalid & $\leq 0$ & -1 \\ \hline
|
||
total\_cost & Valid & $\leq 100$ & 100 \\ \hline
|
||
& Valid & $> 100$ & 101 \\ \hline
|
||
& Invalid & $< 0$ & -2 \\ \hline
|
||
delivery\_time & Valid & NEXT\_DAY, SECOND\_DAY, THIS\_WEEK & SECOND\_DAY \\ \hline
|
||
& Invalid & Null or unsupported value & null \\ \hline
|
||
\end{tabular}}
|
||
\caption{Partitioning for input values with conditions and selected values.}
|
||
\label{tab:partitioning}
|
||
\end{table}
|
||
|
||
\begin{table}[ht]
|
||
\centering
|
||
\resizebox{\columnwidth}{!}{%
|
||
\begin{tabular}{|c|c|c|c|}
|
||
\hline
|
||
\textbf{Total Cost} & \textbf{Number of Items} & \textbf{Shipping Time} & \textbf{Expected Outcome} \\ \hline
|
||
100.0 & 3 & NEXT\_DAY & 25.0 \\ \hline
|
||
150.0 & 3 & SECOND\_DAY & 15.0 \\ \hline
|
||
100.0 & 4 & THIS\_WEEK & 0.0 \\ \hline
|
||
150.0 & 4 & NEXT\_DAY & 30.0 \\ \hline
|
||
100.0 & -1 & NEXT\_DAY & Exception \\ \hline
|
||
-50.0 & 3 & SECOND\_DAY & Exception \\ \hline
|
||
100.0 & 3 & null & Exception \\ \hline
|
||
\end{tabular}}
|
||
\caption{Test cases generated for equivalence testing}
|
||
\label{tab:valid_scenarios}
|
||
\end{table}
|
||
|
||
The total test case generation and implementation took 40 minutes. While running 7 test cases took 0.028 seconds , running the full `mvn test` command took 6.49 seconds.
|
||
|
||
\subsection{Boundary Value Testing}
|
||
|
||
For boundary value testing `delivery\_time` does not eligable since enum's do not have boundaries. For other two variables, values just before, exactly and just above valid input range and logical branching boundaries are selected. Those values are {-1,0,1} for both `number\_of\_items` and `total\_cost` while branching boundary values are {3,4} and {100,101} respectively as can seen in \autoref{boundaries}
|
||
|
||
|
||
\begin{table}[ht]
|
||
\centering
|
||
\begin{tabular}{|l|c|}
|
||
\hline
|
||
\textbf{Input} & \textbf{Boundaries} \\ \hline
|
||
\multirow{5}{*}{number\_of\_items}
|
||
& -1 \\ \cline{2-2}
|
||
& 0 \\ \cline{2-2}
|
||
& 1 \\ \cline{2-2}
|
||
& 3 \\ \cline{2-2}
|
||
& 4 \\ \hline
|
||
\multirow{5}{*}{total\_cost}
|
||
& -1 \\ \cline{2-2}
|
||
& 0 \\ \cline{2-2}
|
||
& 1 \\ \cline{2-2}
|
||
& 100 \\ \cline{2-2}
|
||
& 101 \\ \hline
|
||
delivery\_time & n/a \\ \hline
|
||
\end{tabular}
|
||
\caption{Boundary values for input parameters.}
|
||
\label{boundaries}
|
||
\end{table}
|
||
|
||
\subsubsection{Test Cases}
|
||
Using above boundaries following test scenarios generated.
|
||
\begin{table}[ht]
|
||
\centering
|
||
\resizebox{\columnwidth}{!}{%
|
||
\begin{tabular}{|c|c|c|c|c|}
|
||
\hline
|
||
\textbf{number\_of\_items} & \textbf{total\_cost} & \textbf{delivery\_time} & \textbf{Partition Type} & \textbf{Expected Outcome} \\ \hline
|
||
3 & 100 & NEXT\_DAY & All Valid & 25.00 \\ \hline
|
||
3 & 101 & SECOND\_DAY & All Valid & 15.00 \\ \hline
|
||
4 & 100 & THIS\_WEEK & All Valid & 0.00 \\ \hline
|
||
4 & 101 & NEXT\_DAY & All Valid & 30.00 \\ \hline
|
||
-1 & 100 & NEXT\_DAY & Invalid Items & Exception \\ \hline
|
||
3 & -50 & SECOND\_DAY & Invalid Cost & Exception \\ \hline
|
||
3 & 100 & null & Invalid Delivery & Exception \\ \hline
|
||
\end{tabular}}
|
||
\caption{Test cases generated for boundary value testing}
|
||
\label{tab:test_cases}
|
||
\end{table}
|
||
|
||
The total test case generation and implementation took 24 minutes. While running 9 test cases took 0.026 seconds , running the full `mvn test` command took 7.01 seconds.
|
||
|
||
% Conclusion Section
|
||
\section{Experiment 2}
|
||
\subsection{Decision Table Testing}
|
||
The decision table enumerates various combinations of conditions (employee status, special authorization, auditor status, hour of day, and weekend) and shows the expected outcome. The conditions chosen are representative samples from the decision space.
|
||
|
||
\begin{table}[ht]
|
||
\centering
|
||
\resizebox{\columnwidth}{!}{%
|
||
\begin{tabular}{|c|c|c|c|c|c|c|}
|
||
\hline
|
||
\textbf{\#} & \textbf{isEmployee} & \textbf{hasSpecialAuth} & \textbf{isAuditor} & \textbf{hourOfDay} & \textbf{isWeekend} & \textbf{Expected Result} \\ \hline
|
||
1 & False & False & False & 9 & False & Deny (false) \\ \hline
|
||
2 & True & False & False & 9 & False & Allow (true) \\ \hline
|
||
3 & True & True & False & 8 & False & Allow (true) \\ \hline
|
||
4 & True & False & False & 17 & False & Deny (false) \\ \hline
|
||
5 & True & False & False & 9 & True & Deny (false) \\ \hline
|
||
6 & False & False & True & 9 & True & Allow (true) \\ \hline
|
||
7 & False & False & True & 8 & False & Deny (false) \\ \hline
|
||
8 & True & True & False & 17 & True & Allow (true) \\ \hline
|
||
\end{tabular}}
|
||
\caption{Test cases generated for for decision table testing}
|
||
\label{tab:access_control_test_cases}
|
||
\end{table}
|
||
|
||
The total test case generation and implementation took around 1 hour and 30 minutes. While running above cases took 5709 nanoseconds.
|
||
|
||
\subsection{Equivalence Testing}
|
||
Here, input domains are divided into equivalence classes. We pick one representative test from each class to reduce the total number of tests while still ensuring coverage of all logical categories.
|
||
|
||
\begin{table}[ht]
|
||
\centering
|
||
\resizebox{\columnwidth}{!}{%
|
||
\begin{tabular}{|c|p{8cm}|c|}
|
||
\hline
|
||
\textbf{\#} & \textbf{Conditions (Representative)} & \textbf{Expected Result} \\ \hline
|
||
1 & Not Employee, Not Auditor, Hour=10 (within 9–16), Weekday=false (not weekend) & Deny (false) \\ \hline
|
||
2 & Employee, No Special Auth, Not Auditor, Hour=10 (within 9–16), Weekday=false & Allow (true) \\ \hline
|
||
3 & Employee, Special Auth, Any Hour (e.g., 17), Weekend=true & Allow (true) \\ \hline
|
||
4 & Auditor, Hour=16 (within 9–16), Weekend=true & Allow (true) \\ \hline
|
||
5 & Auditor, Hour=8 (before 9), Weekday=false & Deny (false) \\ \hline
|
||
\end{tabular}}
|
||
\caption{Test cases generated for for equivalence testing}
|
||
\label{tab:representative_conditions}
|
||
\end{table}
|
||
The total test case generation and implementation took around 45 minutes. While running above cases took 3917 nanoseconds.
|
||
|
||
|
||
Equivalence Classes Considered:
|
||
\begin{enumerate}
|
||
\item Employee vs. Non-Employee
|
||
\item Auditor vs. Non-Auditor
|
||
\item Special Authorization vs. No Special Authorization
|
||
\item Inside vs. Outside Working Hours (9–16)
|
||
\item Weekend vs. Weekday
|
||
\end{enumerate}
|
||
|
||
\subsection{Boundary Value Testing}
|
||
|
||
Boundary values are chosen around the critical time limits. For this scenario, critical hours are 8 (just before 9), 9 (start of working hours), 16 (end of working hours), and 17 (just after 16).
|
||
|
||
\begin{table}[ht]
|
||
\centering
|
||
\resizebox{\columnwidth}{!}{%
|
||
\begin{tabular}{|c|p{8cm}|c|}
|
||
\hline
|
||
\textbf{\#} & \textbf{Conditions (Focus on Hour Boundaries)} & \textbf{Expected Result} \\ \hline
|
||
1 & Employee, No Special Auth, Hour=8 (just before start), Weekday=false & Deny (false) \\ \hline
|
||
2 & Employee, No Special Auth, Hour=9 (start), Weekday=false & Allow (true) \\ \hline
|
||
3 & Employee, No Special Auth, Hour=16 (end), Weekday=false & Allow (true) \\ \hline
|
||
4 & Employee, No Special Auth, Hour=17 (just after end), Weekday=false & Deny (false) \\ \hline
|
||
5 & Auditor, Hour=8 (before start), Weekend or Weekday (e.g., false) & Deny (false) \\ \hline
|
||
6 & Auditor, Hour=9 (start), Weekend=true & Allow (true) \\ \hline
|
||
7 & Employee, Special Auth, Hour=8 (any boundary), Weekend=true & Allow (true) \\ \hline
|
||
\end{tabular}}
|
||
\caption{Test cases generated for for boundary value testing}
|
||
\label{tab:hour_boundaries}
|
||
\end{table}
|
||
|
||
The total test case generation and implementation took around 1 hour and 15 minutes. While running above cases took 5876 nanoseconds.
|
||
|
||
|
||
Boundary Values Considered:
|
||
\begin{enumerate}
|
||
\item Hours: 8, 9, 16, 17
|
||
\item Weekend vs. Weekday, Employee vs. Auditor, Special Auth variations
|
||
\end{enumerate}
|
||
|
||
|
||
\section{Conclusion}
|
||
|
||
\onecolumn
|
||
\section*{Additional Resources}
|
||
All of this code and reports and report codes can be fount at \href{https://git.alicanzeybek.xyz/alican/testexperiment}{projects git page}.
|
||
|
||
|
||
\end{document}
|