felix/master_thesis
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

more writing, added benchmark to compare log files

Browse Source
This commit is contained in:
Felix Steghofer
2017-11-30 18:48:23 +01:00
parent c1237406f8
commit 673464137e
8 changed files with 68 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
Thesis/content/Technical_Background/Benchmarks/Benchmarks.tex Normal file
View File

@@ -0,0 +1,22 @@
\section{Benchmarks}
\label{sec:benchmarks}
To get a better understanding of performance related challenges, some benchmarks are performed and described in this section. All benchmarks are performed on the same machine with 16 GB of DD3 RAM with a clock speed of 1600 MT/s in dual channel, an Intel i7-3520M CPU @ 2900 MHz and a Samsung SSD 850 EVO with 250 GB (where not otherwise specified). Linux 4.13.12-1 has been used and Python scripts are executed with Python interpreter in version 3.6.3. For consistency, no other software is running at the time of the benchmark execution (e.g. a desktop environment or heavy background processes) \todo{list of what is running}. All benchmark are run ten times and outliers that show a run time of 10\% above the statistical median are ignored. Although considering the mentioned actions, it is not safe to assume completely equal initial situations at the time of execution on non real-time operating systems (like the one used). So these figures have to be treated with care and should only give a fundamental understanding of how long tasks are about to run.
\begin{lstlisting}[language={bash}, caption={Benchmark: Load and iterate one day of compressed pdns logs}, label={lst:load_and_iterate_one_day_of_compressed_pdns_logs}]
start_z = time.time()
globbed = glob.glob('/home/felix/pdns/' + '*-2017-09-01*.csv.gz')
for f in globbed:
with gzip.open(f, 'rt', newline='') as file:
reader = csv.reader(file)
for row in reader:
pass
print('iterating day took: ' + str(time.time() - start_z) + ' s')
# result:
cleaned results: [155.0667760372162, 148.00951623916626, 147.8429672718048, 147.2554485797882, 147.1039183139801, 147.26967453956604, 147.13052105903625, 147.33162689208984, 147.20316672325134, 147.29751586914062]
average: 148.15111315250397 seconds
\end{lstlisting}

8
Thesis/content/Technical_Background/DNS/DNS.tex
View File

@@ -239,4 +239,10 @@ QType & Type & Description \\
\caption{Address Resolution}
\label{fig:address_resolution}
\end{figure}
\todo{not referenced atm}
\todo{not referenced atm}
\subsection{Passive DNS}
\label{subsec:passive_dns}

3
Thesis/content/Technical_Background/Technical_Background.tex
View File

@@ -2,4 +2,5 @@
\label{cha:technical_background}
\input{content/Technical_Background/DNS/DNS}
\input{content/Technical_Background/Detecting_Malicious_Domain_Names/Detecting_Malicious_Domain_Names}
\input{content/Technical_Background/Detecting_Malicious_Domain_Names/Detecting_Malicious_Domain_Names}
\input{content/Technical_Background/Benchmarks/Benchmarks}