Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactored solc --link to use LinkerObject #12756

Closed
wants to merge 67 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
d67f667
Refactored solc --link to use LinkerObject
Mar 8, 2022
fb79966
Addressed comments on PR
Mar 9, 2022
ee3fa6c
fixed test failures
Mar 9, 2022
885a851
added tests, refactored CommandLineInterface::link to be static
Mar 10, 2022
0d8a6dc
added additional test and refactored so can handle all placeholder co…
Mar 22, 2022
08be5d5
fixed broken tests
Mar 22, 2022
110aadf
[CMake] Refactor libsolc linker flag for EXPORTED_FUNCTIONS and make …
christianparpart Mar 7, 2022
95e5827
Simplify peephole optimizer template.
chriseth Mar 9, 2022
61dc66d
Store whether an evmasm Assembly is creation code.
ekpyron Mar 7, 2022
6f88399
Add test case and adjust tests.
ekpyron Mar 7, 2022
eeaf62c
Adjust documentation.
ekpyron Mar 7, 2022
427ae25
Remove isCreation flag from OptimiserSettings.
ekpyron Mar 7, 2022
246d06b
Unit test.
ekpyron Mar 9, 2022
fe9a98b
Disable flaky ElementFi tests
cameel Feb 28, 2022
884601c
Update gnosis external test to use upstream directly and use Hardhat
cameel Oct 26, 2021
76a7f66
CI: Remove the compile-only run for Gnosis
cameel Jan 18, 2022
9611534
doc: improve explanation of interfaces
kanedaaaa Mar 9, 2022
8ae30f1
Remove unneeded header files.
chriseth Mar 10, 2022
df1ac88
euler: Override Hardhat test timeout
cameel Mar 9, 2022
05cecd3
Provide generic access to type definition for user-defined types.
chriseth Mar 9, 2022
a018cce
Optimize eq iszero jumpi to xor jumpi and remove double jump.
chriseth Mar 7, 2022
dc0cd5d
Unused store eliminator.
chriseth May 5, 2021
de0234e
euler: Use project's own TEST_TIMEOUT var for changing the timeout in…
cameel Mar 10, 2022
143e57e
Error messages from abi encodecall are giving more details about the …
nishant-sachdeva Mar 11, 2022
9438774
Remove use of `using namespace` in header file
tylerfxn Mar 7, 2022
ae86b23
Specify namespaces
tylerfxn Mar 7, 2022
02587af
more qualifying
tylerfxn Mar 10, 2022
2abd612
Fix CommonSyntaxTest.cpp and others
tylerfxn Mar 10, 2022
4934de6
Re-enable running tests in external tests via IR in cases where they …
cameel Mar 1, 2022
cbb34f0
Generate warning when rational numbers are converted to their mobile …
nishant-sachdeva Feb 22, 2022
5ca531f
Use "C" instead user environment locale in solc
Feb 25, 2022
17520b3
A script and CI job for testing the compiler with different locales
cameel Feb 25, 2022
1f953ca
Change memory-safe assembly annotation suggestion during stack too de…
ekpyron Mar 14, 2022
78a63c2
LSP: Introduce HandlerBase for future LSP-feature implementations.
christianparpart Jan 5, 2022
e612b03
LSP: Implements goto-definition.
christianparpart Dec 20, 2021
4da987d
Clean up changelog.
chriseth Mar 14, 2022
ac2562b
Extend using-for.
hrkrshnn Oct 11, 2021
6cf3719
Add not_existing_import.json to ASTJSON tests
Feb 17, 2022
ee98d16
Fix syntax errors in ASTJSON tests
Feb 17, 2022
bf05321
Non-fatal erros do not stop ASTJSON test execution
Feb 18, 2022
544dcb9
Support compilation fail scenarios in ASTJSON tests
Feb 22, 2022
565c951
Add "fail after parsing" scenario to ASTJSON tests
Feb 23, 2022
701f61d
Generate ASTJSONTest variants only if a file with expected result exists
Feb 23, 2022
47742df
Use sol file to validate test variants
wechman Mar 7, 2022
250b60b
Mark viaIR code generation as non-experimental.
ekpyron Mar 9, 2022
7626442
No longer mark Yul as experimental.
ekpyron Mar 14, 2022
c48166b
Fix import directive visits in type checker and view pure checker.
ekpyron Mar 11, 2022
198da88
Permit control flow mutations in global scope of a Yul program.
bshastry Mar 14, 2022
e7f8ba9
Using for with global binding.
chriseth Nov 16, 2021
0d96f70
fix: corrects _ prefixes
devtooligan Feb 16, 2022
3ee5125
Test for old behaviour.
ekpyron Mar 10, 2022
94a7308
Fix and updated test.
ekpyron Mar 10, 2022
0df4a8b
Using for at file level.
chriseth Mar 15, 2022
492bcfc
Add helper to see if a switch has a default case.
chriseth Mar 15, 2022
4fd60b6
Refactor data flow analyzer state access.
chriseth Mar 10, 2022
680acd0
Create a state struct.
chriseth Mar 10, 2022
90a8993
Test case for current behaviour.
ekpyron Jan 12, 2022
06ace46
Fill in junk in stack layouts on terminating control flow paths.
ekpyron Jan 14, 2022
f193f43
Set release date.
chriseth Mar 16, 2022
7feeaa8
Move state variable initialization to top.
chriseth Mar 16, 2022
c0483ff
CommonSubexpressionEliminator performance optimization
wechman Mar 16, 2022
2b52dfc
Set version to 0.8.14.
chriseth Mar 16, 2022
b4b0188
Refactor stack compressor.
chriseth Mar 15, 2022
d0567f8
Simplify rematerialization candidates.
chriseth Mar 16, 2022
6fda772
Fixed ambiguity in the create2 address computation doc
hroussille Mar 16, 2022
034bdc6
Additional peephole optimizer rules for removing side-effect free ins…
ekpyron Mar 15, 2022
212b2b4
Test updates.
ekpyron Mar 15, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
161 changes: 80 additions & 81 deletions solc/CommandLineInterface.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -654,8 +654,7 @@ void CommandLineInterface::processInput()
assemble(m_options.assembly.inputLanguage, m_options.assembly.targetMachine);
break;
case InputMode::Linker:
solAssert(m_options.input.mode == InputMode::Linker, "");
link(m_fileReader, m_options.linker.libraries, m_hasOutput, m_serr);
link(m_fileReader, m_options.linker.libraries);
writeLinkedFiles();
break;
case InputMode::Compiler:
Expand Down Expand Up @@ -918,12 +917,64 @@ void CommandLineInterface::serveLSP()
solThrow(CommandLineExecutionError, "LSP terminated abnormally.");
}

static bool isBytecodeChar(char c)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think keeping this helper may still be nice.

static bool isBytecodeString(string s)
{
return isxdigit(c) || c == '_' || c == '$';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated to this PR, but:

isxdigit(std::locale) | checks if a character is classified as a hexadecimal digit by a locale(function template)

Perhaps we'd be better off with just a manual check and could make it constexpr that case.

if (s.empty())
return false;

size_t placeholderAddressStart = s.find("__");
if (placeholderAddressStart == string::npos)
return false;
placeholderAddressStart += 3;

size_t placeholderAddressEnd = s.find("__", placeholderAddressStart + 1) - 2;

for (size_t i = 0; i < s.size(); i++)
{
// inside the markers can contain anything, outside the hex digits is hex characters and spaces only
if (i >= placeholderAddressStart && i <= placeholderAddressEnd)
continue;

if (!isxdigit(s[i]) && s[i] != ' ' && s[i] != '$' && s[i] != '_')
return false;
}
return true;
}

void CommandLineInterface::link(frontend::FileReader& fileReader, map<string, h160> const& libraries, bool& hasError, ostream& serr)
static size_t validateBytecode(string const& s, string const& fileName, string const& openingMarker, string const& closingMarker)
{
size_t firstMarker = s.find(openingMarker);
size_t lastMarker = firstMarker + (openingMarker.size() - 1);

while (s.find(closingMarker, lastMarker) != string::npos)
lastMarker = s.find(closingMarker, lastMarker) + 1;

if (lastMarker == firstMarker + (openingMarker.size() - 1))
solThrow(
CommandLineExecutionError,
"Error in binary object file " + fileName + " unbounded placeholder at " + to_string(firstMarker) + "\n"
);

lastMarker = lastMarker + openingMarker.size() - 1;

if ((lastMarker - firstMarker) != 40)
{
solThrow(
CommandLineExecutionError,
"Error in binary object file \"" + fileName + "\" placeholder of invalid length found in column " + to_string(firstMarker) + ".\n"
+ " Placeholders must be exactly 40 characters long, including address markers __$ and $__.\n"
);
}

return firstMarker;
}

static size_t validateBytecode(string const& s, string const& fileName, string const& openingMarker)
{
return validateBytecode(s, fileName, openingMarker, openingMarker);
}

void CommandLineInterface::link(frontend::FileReader& fileReader, map<string, h160> const& libraries)
{
// Map from how the libraries will be named inside the bytecode to their addresses.
map<string, h160> librariesReplacements;
Expand All @@ -938,103 +989,51 @@ void CommandLineInterface::link(frontend::FileReader& fileReader, map<string, h1

for (auto& src: sourceCodes)
{
// Map from how the libraries will be named inside the bytecode to their addresses.
// Map from how the libraries will be named inside the bytecodeIter to their addresses.
map<size_t, string> linkReferences;
string bytecode;
vector<string> sourceLines;
boost::split(sourceLines, src.second, boost::is_any_of("\n"));

size_t bytecodeIndex;
bool bytecodeFound = false;
for (size_t i = 0; i < sourceLines.size(); ++i)
{
// If all characters are bytecode characters
if (sourceLines[i].size() > 0
&& all_of(sourceLines[i].begin(), sourceLines[i].end(), isBytecodeChar))
{
bytecodeFound = true;
bytecodeIndex = i;
break;
}
}

if (!bytecodeFound)
if (src.second.empty())
// no bytecode
continue;

vector<string> sourceLines;
boost::split(sourceLines, src.second, boost::is_any_of("\n"));
auto bytecodeIter = find_if(sourceLines.begin(), sourceLines.end(), isBytecodeString);

for (auto const& library: libraries)
{
string const& name = library.first;
// Bytecode we need is on the third line there's a blank line at the start
bytecode = sourceLines.at(bytecodeIndex);
size_t libraryStartOffset = bytecode.find("__$");
size_t libraryEndOffset = bytecode.find("$__");

bool startMarkerFound = libraryStartOffset != string::npos;
bool endMarkerFound = libraryEndOffset != string::npos;
size_t startMarkerOffset = string::npos;
if (bytecodeIter->find("__$__") != string::npos)
startMarkerOffset = validateBytecode(*bytecodeIter, src.first, "__$__");

if (!startMarkerFound && !endMarkerFound)
{
// No placeholders found
continue;
}
if (bytecodeIter->find("__$") != string::npos)
startMarkerOffset = validateBytecode(*bytecodeIter, src.first, "__$", "$__");

if (startMarkerFound && !endMarkerFound)
{
solThrow(
CommandLineExecutionError,
"Error in binary object file " + src.first + " unbounded placeholder at " + to_string(libraryStartOffset) + "\n"
);
}
if (bytecodeIter->find("__") != string::npos)
startMarkerOffset = validateBytecode(*bytecodeIter, src.first, "__");

if (!startMarkerFound && endMarkerFound)
{
solThrow(
CommandLineExecutionError,
"Error in binary object file " + src.first + " unbounded placeholder without start point __$ at " + to_string(libraryEndOffset) + "\n"
);
}

if (startMarkerFound && endMarkerFound)
{
if ((libraryEndOffset - libraryStartOffset) != placeholderSize - 3)
{
solThrow(
CommandLineExecutionError,
"Error in binary object file " + src.first + " truncated placeholder found at " + to_string(libraryStartOffset) +
" placeholder addresses should be " + to_string(placeholderSize) + " characters long (including address markers __$ and $__)\n"
);
}
}

string foundPlaceholder;
if (startMarkerOffset == string::npos)
// no markers found
continue;

for (size_t i = 0; i < placeholderSize; i++)
{
foundPlaceholder += bytecode[libraryStartOffset + i];
bytecode[libraryStartOffset + i] = '0';
}
string foundPlaceholder = bytecodeIter->substr(startMarkerOffset,placeholderSize);
bytecodeIter->replace(startMarkerOffset, placeholderSize, "0000000000000000000000000000000000000000");

if (libraryStartOffset != bytecode.length())
if (startMarkerOffset != bytecodeIter->length())
{
// Divide by 2 because it's going to be converted to bytes
linkReferences[libraryStartOffset / 2] = name;
linkReferences[startMarkerOffset / 2] = name;
}
else
{
serr << "Reference \"" << foundPlaceholder << "\" in file \"" << src.first << "\" still unresolved."<< endl;
hasError = true;
}
serr() << "Reference \"" << foundPlaceholder << "\" in file \"" << src.first << "\" still unresolved."<< endl;
}

evmasm::LinkerObject linkerObject = {
fromHex(bytecode),
linkReferences,
{},
{}
};
evmasm::LinkerObject linkerObject = {fromHex(*bytecodeIter),linkReferences,{},{}};

linkerObject.link(librariesReplacements);
sourceLines.at(bytecodeIndex) = toHex(linkerObject.bytecode);
bytecodeIter->replace(bytecodeIter->begin(), bytecodeIter->end(), toHex(linkerObject.bytecode));

string resolvedBytecode = boost::algorithm::join(sourceLines, "\n");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not just bytecode. It's the whole file reassembled.

Suggested change
string resolvedBytecode = boost::algorithm::join(sourceLines, "\n");
string resolvedSrc = boost::algorithm::join(sourceLines, "\n");

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if it's a good idea to rewrite the newlines. At least it changes the current behavior because the previous implementation would preserve them.

src.second = resolvedBytecode;
Expand Down
2 changes: 1 addition & 1 deletion solc/CommandLineInterface.h
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ class CommandLineInterface
FileReader const& fileReader() const { return m_fileReader; }
std::optional<std::string> const& standardJsonInput() const { return m_standardJsonInput; }

static void link(frontend::FileReader& fileReader, std::map<std::string, util::h160> const& libraries, bool& hasError, std::ostream& serr);
void link(frontend::FileReader& fileReader, std::map<std::string, util::h160> const& libraries);

private:
void printVersion();
Expand Down
Loading