Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Push 2024 06 30 #770

Merged
merged 44 commits into from
Jul 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
5cf322b
add service file for build.opensuse.org
rdementi Jun 21, 2024
4bf7325
add PCM_HARDENING_FLAGS
rdementi Jun 21, 2024
211e11d
add -fstack-protector option
rdementi Jun 21, 2024
af97dd1
add -D_FORTIFY_SOURCE=2
rdementi Jun 21, 2024
5e4c279
added corpus for urltest-fuzz
rdementi Jun 22, 2024
b6f2b67
use the corpus and refactor fuzz.sh
rdementi Jun 22, 2024
1692c5e
add fuzzer for pcm-sensor-server
rdementi Jun 24, 2024
5ad7a93
replace files with human readable corpus
rdementi Jun 24, 2024
8bd3024
print errno string
rdementi Jun 24, 2024
f818668
recv may fail sometimes
rdementi Jun 24, 2024
d62a429
fix 'invalid format character' error
rdementi Jun 24, 2024
fdca530
the scripts require bash. call bash explicitly
rdementi Jun 25, 2024
637119d
ignore leaks on libFuzzer side use
rdementi Jun 25, 2024
7ac4e31
fix a memory leak and throw exception with verbose reason
rdementi Jun 25, 2024
f8dda06
fix heap-buffer-overflow
rdementi Jun 25, 2024
e8219da
add /sys/fs/cgroup/cpuset/cpuset.cpus diagnostic message in tests (fo…
rdementi Jun 25, 2024
14dc7ac
make fuzzing period configurable
rdementi Jun 26, 2024
00369a0
make the fuzz-job run long on saturdays
rdementi Jun 26, 2024
78bda31
add fuzz-short-job
rdementi Jun 26, 2024
cc5d6e2
add on-push/pr micro fuzzing job
rdementi Jun 26, 2024
8c7d1cf
Stop using deprecated functions (#643)
ogbrugge-work Jun 27, 2024
f1c7703
fix a memory leak in SSL code
rdementi Jun 26, 2024
c4a205a
set pointers to nullptr after delete or free
rdementi Jun 28, 2024
4546df0
address warning: nullptr check
rdementi Jun 28, 2024
14e51ab
cmake: print more flags
rdementi Jun 28, 2024
3cb171b
more hardening compile flags
rdementi Jun 28, 2024
d103314
fix compilation on old gcc
rdementi Jun 28, 2024
2aff41b
do not include shared library code into pcm-core executable
rdementi Jun 28, 2024
4fe5318
delete forbidden methods
rdementi Jun 28, 2024
6cbab91
fix linking with asan
rdementi Jun 28, 2024
f1bd4a1
use compiled lib, not the installed one
rdementi Jun 29, 2024
8cf0a33
avoid pre-mature PCM init triggered by global counter state init
rdementi Jun 29, 2024
cf9f5d2
workaround a known bug with sanitizers
rdementi Jun 29, 2024
17c10e4
add -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer
rdementi Jun 29, 2024
6a6ce98
harden the debug build too
rdementi Jun 29, 2024
98d9f98
fix compilation on gcc48
rdementi Jun 29, 2024
dc7f402
pcm-latency: avoid pre-mature PCM init triggered by global counter st…
rdementi Jun 29, 2024
4b92b80
run long fuzzer every every sunday at midnight
rdementi Jun 30, 2024
c461cd1
increase rss limit
rdementi Jun 30, 2024
65cfffa
reduce the duration fuzzing CI job to fit the GH timeout budget
rdementi Jun 30, 2024
aa549b1
parameter value checks
rdementi Jun 30, 2024
f2f062a
Merge tag '2024-06-30' into push-2024-06-30
rdementi Jul 1, 2024
ee69c2c
try to resolve link error on OSX
rdementi Jul 1, 2024
eaaae6d
try to fix compilation on *BSD
rdementi Jul 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .github/workflows/ci-fuzz-micro.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: fuzz-micro-job

on:
push:
branches: [ '**' ]
pull_request:
branches: [ '**' ]

permissions:
contents: read

jobs:
fuzz:
runs-on: ci-test
if: ${{ github.repository != 'intel/pcm' }}

steps:
- name: Harden Runner
uses: step-security/harden-runner@eb238b55efaa70779f274895e782ed17c84f2895 # v2.6.1
with:
egress-policy: audit

- uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3.6.0
with:
submodules: recursive

- name: Build and test
run: |
cmake --version
set -o pipefail
mkdir build
cd build
bash ${{ github.workspace }}/tests/fuzz.sh 5 2>&1 | tee fuzz-log.txt
cd ..

- name: Show report
run: |
cat build/report.txt
echo "Fuzzing completed"

- name: upload-artifact
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
with:
name: fuzz-log-${{ github.sha }}
path: "build/fuzz-log.txt"


46 changes: 46 additions & 0 deletions .github/workflows/ci-fuzz-short.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: fuzz-short-job

on:
# manual triggering
workflow_dispatch:


permissions:
contents: read

jobs:
fuzz:
runs-on: ci-test
if: ${{ github.repository != 'intel/pcm' }}

steps:
- name: Harden Runner
uses: step-security/harden-runner@eb238b55efaa70779f274895e782ed17c84f2895 # v2.6.1
with:
egress-policy: audit

- uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3.6.0
with:
submodules: recursive

- name: Build and test
run: |
cmake --version
set -o pipefail
mkdir build
cd build
bash ${{ github.workspace }}/tests/fuzz.sh 10 2>&1 | tee fuzz-log.txt
cd ..

- name: Show report
run: |
cat build/report.txt
echo "Fuzzing completed"

- name: upload-artifact
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
with:
name: fuzz-log-${{ github.sha }}
path: "build/fuzz-log.txt"


2 changes: 1 addition & 1 deletion .github/workflows/ci-fuzz.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
set -o pipefail
mkdir build
cd build
sh ${{ github.workspace }}/tests/fuzz.sh 2>&1 | tee fuzz-log.txt
bash ${{ github.workspace }}/tests/fuzz.sh 300 2>&1 | tee fuzz-log.txt
cd ..

- name: Show report
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:
- name: Test
run: |
set -o pipefail
sh ${{ github.workspace }}/tests/test.sh 2>&1 | tee test-log.txt
bash ${{ github.workspace }}/tests/test.sh 2>&1 | tee test-log.txt

- name: upload-artifact
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
Expand Down
19 changes: 16 additions & 3 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -82,9 +82,18 @@ if(UNIX) # APPLE, LINUX, FREE_BSD
elseif()
set (PCM_DYNAMIC "")
endif()
set(CMAKE_CXX_FLAGS_RELEASE "${PCM_OPTIONAL_FLAGS} -O3 ${PCM_DYNAMIC}")
set(CMAKE_CXX_FLAGS_DEBUG "${PCM_OPTIONAL_FLAGS} -O0 -g ${PCM_DYNAMIC}")
set(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${PCM_OPTIONAL_FLAGS} -O3 -g ${PCM_DYNAMIC}")
set(PCM_HARDENING_FLAGS "-fPIE -fstack-protector -D_FORTIFY_SOURCE=2 -ftrapv -fsanitize=address -fwrapv -fno-delete-null-pointer-checks -fno-strict-overflow -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer")
if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 5)
message(WARNING "Old gcc compiler (version < 5), -fsanitize=undefined option is not supported.")
elseif()
set(PCM_HARDENING_FLAGS "${PCM_HARDENING_FLAGS} -fsanitize=undefined")
endif()
set(PCM_LINKER_HARDENING_FLAGS "-fsanitize=address")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${PCM_LINKER_HARDENING_FLAGS}")
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${PCM_LINKER_HARDENING_FLAGS}")
set(CMAKE_CXX_FLAGS_RELEASE "${PCM_OPTIONAL_FLAGS} -O3 ${PCM_DYNAMIC} ${PCM_HARDENING_FLAGS}")
set(CMAKE_CXX_FLAGS_DEBUG "${PCM_OPTIONAL_FLAGS} -O0 -g ${PCM_DYNAMIC} ${PCM_HARDENING_FLAGS}")
set(CMAKE_CXX_FLAGS_RELWITHDEBINFO "${PCM_OPTIONAL_FLAGS} -O3 -g ${PCM_DYNAMIC} ${PCM_HARDENING_FLAGS}")
if(FREE_BSD)
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -lexecinfo")
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -lexecinfo")
Expand All @@ -94,6 +103,10 @@ if(UNIX) # APPLE, LINUX, FREE_BSD
set(CMAKE_C_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG}")
set(CMAKE_C_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO}")

message(STATUS "CMAKE_CXX_FLAGS_RELEASE: ${CMAKE_CXX_FLAGS_RELEASE}")
message(STATUS "CMAKE_CXX_FLAGS_DEBUG: ${CMAKE_CXX_FLAGS_DEBUG}")
message(STATUS "CMAKE_CXX_FLAGS_RELWITHDEBINFO: ${CMAKE_CXX_FLAGS_RELWITHDEBINFO}")

endif(UNIX)

if(FUZZ)
Expand Down
12 changes: 12 additions & 0 deletions _service
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
<services>
<service name="download_url">
<param name="host">github.com</param>
<param name="protocol">https</param>
<param name="path">/intel/pcm/archive/master.zip</param>
</service>
<service name="download_url">
<param name="host">github.com</param>
<param name="protocol">https</param>
<param name="path">/intel/pcm/archive/master.tar.gz</param>
</service>
<service name="download_url"><param name="host">raw.githubusercontent.com</param><param name="protocol">https</param><param name="path">/intel/pcm/master/pcm.spec</param></service></services>
19 changes: 16 additions & 3 deletions src/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright (c) 2022, Intel Corporation

include(FindOpenSSL)

# All pcm-* executables
set(PROJECT_NAMES pcm pcm-numa pcm-latency pcm-power pcm-msr pcm-memory pcm-tsx pcm-pcie pcm-core pcm-iio pcm-lspci pcm-pcicfg pcm-mmio pcm-tpmi pcm-raw pcm-accel)

set(MINIMUM_OPENSSL_VERSION 1.1.1)

file(GLOB COMMON_SOURCES pcm-accel-common.cpp msr.cpp cpucounters.cpp pci.cpp mmio.cpp tpmi.cpp pmt.cpp bw.cpp utils.cpp topology.cpp debug.cpp threadpool.cpp uncore_pmu_discovery.cpp)

if (APPLE)
Expand Down Expand Up @@ -32,8 +35,11 @@ if(UNIX) # LINUX, FREE_BSD, APPLE
$<$<CONFIG:RelWithDebInfo>:PCM_SILENT>
)

target_compile_options(PCM_STATIC_SILENT PRIVATE "-fPIC")

# libpcm.so
add_library(PCM_SHARED SHARED pcm-core.cpp)
target_compile_options(PCM_SHARED PRIVATE -DPCM_SHARED_LIBRARY=1)
# PCM_SILENT in Release* for pcm-core.cpp
target_compile_definitions(PCM_SHARED PRIVATE
$<$<CONFIG:Release>:PCM_SILENT>
Expand All @@ -44,6 +50,8 @@ if(UNIX) # LINUX, FREE_BSD, APPLE
add_subdirectory(MacMSRDriver)
include_directories("${CMAKE_SOURCE_DIR}/src/MacMSRDriver") # target_include_directories doesn't work
target_link_libraries(PCM_SHARED PRIVATE PCM_STATIC_SILENT PcmMsr Threads::Threads)
elseif(LINUX)
target_link_libraries(PCM_SHARED PRIVATE PCM_STATIC_SILENT Threads::Threads asan)
else()
target_link_libraries(PCM_SHARED PRIVATE PCM_STATIC_SILENT Threads::Threads)
endif()
Expand Down Expand Up @@ -145,9 +153,14 @@ foreach(PROJECT_NAME ${PROJECT_NAMES})
else()
message(STATUS "Compiling with SSL support, requires libssl-dev or openssl-devel or libopenssl-devel or libopenssl-dev package installed")
message(STATUS "To disable SSL support, use -DNO_SSL=1 option")
find_package(OpenSSL REQUIRED)
target_compile_options(${PROJECT_NAME} PRIVATE "-DUSE_SSL")
set(LIBS ${LIBS} OpenSSL::SSL OpenSSL::Crypto)
find_package(OpenSSL ${MINIMUM_OPENSSL_VERSION} QUIET)
if(OPENSSL_FOUND)
message(STATUS "OpenSSL version ${OPENSSL_VERSION} >= ${MINIMUM_OPENSSL_VERSION}, OpenSSL support enabled")
target_compile_options(${PROJECT_NAME} PRIVATE "-DUSE_SSL")
set(LIBS ${LIBS} OpenSSL::SSL OpenSSL::Crypto)
else()
message(STATUS "OpenSSL support has been disabled, the version is less than ${MINIMUM_OPENSSL_VERSION}")
endif()
endif()
file(READ pcm-sensor-server.service.in SENSOR_SERVICE_IN)
string(REPLACE "@@CMAKE_INSTALL_SBINDIR@@" "${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_SBINDIR}" SENSOR_SERVICE "${SENSOR_SERVICE_IN}")
Expand Down
8 changes: 4 additions & 4 deletions src/cpuasynchcounter.h
Original file line number Diff line number Diff line change
Expand Up @@ -77,10 +77,10 @@ class AsynchronCounterState {
{
std::cerr << "PCM Error in ~AsynchronCounterState(). Exception " << e.what() << "\n";
}
delete[] cstates1;
delete[] cstates2;
delete[] skstates1;
delete[] skstates2;
deleteAndNullifyArray(cstates1);
deleteAndNullifyArray(cstates2);
deleteAndNullifyArray(skstates1);
deleteAndNullifyArray(skstates2);
}

uint32 getNumCores()
Expand Down
42 changes: 32 additions & 10 deletions src/cpucounters.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -513,7 +513,7 @@ bool PCM::isRDTDisabled() const
flag = 0;
}
#ifdef _MSC_VER
free(env);
freeAndNullify(env);
#endif
}
return flag > 0;
Expand Down Expand Up @@ -1204,7 +1204,7 @@ bool PCM::discoverSystemTopology()

while (res == FALSE)
{
delete[] slpi;
deleteAndNullifyArray(slpi);

if (GetLastError() == ERROR_INSUFFICIENT_BUFFER)
{
Expand Down Expand Up @@ -1263,7 +1263,7 @@ bool PCM::discoverSystemTopology()
socketIdMap[entry.socket] = 0;
}

delete[] base_slpi;
deleteAndNullifyArray(base_slpi);

#else
// for Linux, Mac OS, FreeBSD and DragonFlyBSD
Expand Down Expand Up @@ -1374,7 +1374,7 @@ bool PCM::discoverSystemTopology()
return false; \
} \
ret_value = convertUnknownToInt(size, pParam); \
free(pParam); \
freeAndNullify(pParam); \
}
// End SAFE_SYSCTLBYNAME

Expand Down Expand Up @@ -3307,7 +3307,7 @@ PCM::~PCM()
{
destroyMSR();
instance = NULL;
delete systemTopology;
deleteAndNullify(systemTopology);
}
}

Expand Down Expand Up @@ -7160,7 +7160,7 @@ PciHandleType * ServerUncorePMUs::createIntelPerfMonDevice(uint32 groupnr_, int3

if(vendor_id == PCM_INTEL_PCI_VENDOR_ID) return handle;

delete handle;
deleteAndNullify(handle);
}
return NULL;
}
Expand Down Expand Up @@ -8600,6 +8600,18 @@ ServerUncorePMUs::~ServerUncorePMUs()

void ServerUncorePMUs::programServerUncoreMemoryMetrics(const ServerUncoreMemoryMetrics & metrics, const int rankA, const int rankB)
{
switch (metrics)
{
case PartialWrites:
case Pmem:
case PmemMemoryMode:
case PmemMixedMode:
break;
default:
std::cerr << "PCM Error: unknown memory metrics: " << metrics << "\n";
return;
}

PCM * pcm = PCM::getInstance();
uint32 MCCntConfig[4] = {0,0,0,0};
uint32 EDCCntConfig[4] = {0,0,0,0};
Expand Down Expand Up @@ -8695,6 +8707,16 @@ void ServerUncorePMUs::programServerUncoreMemoryMetrics(const ServerUncoreMemory
}
}
} else {
if (rankA < 0 || rankA > 7)
{
std::cerr << "PCM Error: invalid rankA value: " << rankA << "\n";
return;
}
if (rankB < 0 || rankB > 7)
{
std::cerr << "PCM Error: invalid rankB value: " << rankB << "\n";
return;
}
switch(cpu_model)
{
case PCM::IVYTOWN:
Expand Down Expand Up @@ -9742,7 +9764,7 @@ PciHandleType * getDeviceHandle(uint32 vendorId, uint32 deviceId)
const uint32 did = (value >> 16) & 0xffff;
if (vid == vendorId && did == deviceId)
return h;
delete h;
deleteAndNullify(h);
}
}
}
Expand Down Expand Up @@ -9792,7 +9814,7 @@ uint32 PCM::getMaxNumOfCBoxesInternal() const
num = (uint32)weight32(value);
h->read32(0xa0, &value);
num += (uint32)weight32(value);
delete h;
deleteAndNullify(h);
}
else
{
Expand Down Expand Up @@ -10457,8 +10479,8 @@ CounterWidthExtender::CounterWidthExtender(AbstractRawCounter * raw_counter_, ui
}
CounterWidthExtender::~CounterWidthExtender()
{
delete UpdateThread;
if (raw_counter) delete raw_counter;
deleteAndNullify(UpdateThread);
deleteAndNullify(raw_counter);
}


Expand Down
4 changes: 2 additions & 2 deletions src/daemon/daemon.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -89,8 +89,8 @@ namespace PCMDaemon {

Daemon::~Daemon()
{
delete[] serverUncoreCounterStatesBefore_;
delete[] serverUncoreCounterStatesAfter_;
deleteAndNullifyArray(serverUncoreCounterStatesBefore_);
deleteAndNullifyArray(serverUncoreCounterStatesAfter_);
}

void Daemon::setupPCM()
Expand Down
2 changes: 1 addition & 1 deletion src/memoptest.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ int main(int argc, char * argv[])
cout << "Bandwidth: " << (sizeof(T) * nelements * niter) / ((after_ts - before_ts) * 1024 * 1024) << " MByte/sec\n" << std::flush;
}

delete[] vector;
deleteAndNullifyArray(vector);

return 0;
}
6 changes: 3 additions & 3 deletions src/msr.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -126,8 +126,7 @@ MsrHandle::~MsrHandle()
MsrHandle::num_handles--;
if (MsrHandle::num_handles == 0)
{
delete driver;
driver = NULL;
deleteAndNullify(driver);
}
}

Expand Down Expand Up @@ -228,14 +227,15 @@ MsrHandle::MsrHandle(uint32 cpu) : fd(-1), cpu_id(cpu)
writesEnabled = true;
}
char * path = new char[200];
if (!path) throw std::runtime_error("Allocation of 200 bytes failed.");
snprintf(path, 200, "/dev/cpu/%d/msr", cpu);
int handle = ::open(path, O_RDWR);
if (handle < 0)
{ // try Android msr device path
snprintf(path, 200, "/dev/msr%d", cpu);
handle = ::open(path, O_RDWR);
}
delete[] path;
deleteAndNullifyArray(path);
if (handle < 0)
{
std::cerr << "PCM Error: can't open MSR handle for core " << cpu << " (" << strerror(errno) << ")\n";
Expand Down
Loading
Loading