diff --git a/Ghidra/Configurations/Public_Release/src/global/docs/ChangeHistory.md b/Ghidra/Configurations/Public_Release/src/global/docs/ChangeHistory.md index 9dbda839e5..55bfee889b 100644 --- a/Ghidra/Configurations/Public_Release/src/global/docs/ChangeHistory.md +++ b/Ghidra/Configurations/Public_Release/src/global/docs/ChangeHistory.md @@ -1,29 +1,54 @@ +# Ghidra 12.0.3 Change History (February 2026) + +### New Features +* _Listing_. In order to mitigate possible security risks, auto comments will not longer render annotations in such a way as to make them valid annotation links. Normal comments will continue to work as usual. (GP-6414) + +### Improvements +* _Demangler_. The __Demangler GNU__ analyzer now has a timeout option. (GP-6408) +* _GUI_. Corrected Ghidra GUI to fail-fast in headless environment and avoid stack traces. (GP-6399) +* _Listing_. The `@execute` annotation is no longer supported. (GP-6413) + +### Bugs +* _Data Types_. Corrected multi-user merge issues related to non-packed structures which could negatively affect merge results. (GP-6320, Issue #8776) +* _Debugger_. Fixed a `NullPointerException` that could occur upon closing the Debugger. (GP-6376) +* _Debugger:Breakpoints_. Fixed an issue where restarting a target (e.g., the `run` command from GDB's CLI) caused duplicate breakpoint entries and GUI glitches. (GP-6027) +* _Decompiler_. Fixed _"PTRSUB off of non structured pointer type"_ exceptions caused by `void *` data-type. (GP-6388, Issue #8887) +* _Decompiler_. Fixed source of _"Forced merge caused intersection"_ exceptions when decompiling optimized string copies. (GP-6393, Issue #8651) +* _Multi-User_. Revised Ghidra Server self-signed certificate generation to include all associated FQDNs and IP addresses as subject alternative names. This will address the forced hostname check imposed with the release of JDK 21.0.10. To benefit from this change the Ghidra Server will need to be upgraded to this release. A client-side workaround is to set the following JVM property within `support/launch.properties` by adding the line: `VMARGS=-Djdk.rmi.ssl.client.enableEndpointIdentification=false`. (GP-6426, Issue #8940) +* _Processors_. Fixed bug in AARCH64 `sha1h` instruction to shift instead of rotate bits. (GP-4501, Issue #6398) +* _Processors_. Fixed 80251 disassembly errors for instructions referencing the SPX register. (GP-5905, Issue #8395) +* _Processors_. Fixed disassembly of MIPS16e2 `lui` instruction to only parse on extended words. (GP-6419) +* _Search_. Fixed a memory leak in the `Find References...` action. (GP-6395, Issue #8921) + +### Notable API Changes +* _Data Types_. (GP-6320) Structure offset-based insert methods `Structure.insertAtOffset` will now skip forward over existing zero-length components at the insert offset before performing insert of new component. + # Ghidra 12.0.2 Change History (January 2026) ### New Features -* _Emulator_. Fixed emulator's evaluation of `inst_next2` (GP-6134, Issue #8646) +* _Emulator_. Fixed the Emulator's evaluation of `inst_next2`. (GP-6134, Issue #8646) ### Improvements * _Basic Infrastructure_. Upgraded `commons-lang3` , `log4j`, and `postgresql` jars. (GP-6243) -* _Debugger_. Several Address and Value columns are now displayed in fixed-width font: Register Value, Stack PC, Snapshot PC, Watch Value (GP-6025) +* _Debugger_. Several Address and Value columns are now displayed in fixed-width font: Register Value, Stack PC, Snapshot PC, and Watch Value. (GP-6025) * _Debugger:Breakpoints_. Added __Expression__ column to __Breakpoints__ locations table. (GP-6026) * _Documentation_. Updated Debugger tutorial to reflect the addition of the Comment column to the Watches panel, and the moving of the schedule display to trace tabs instead of the Threads panel title bar. (GP-6032) * _Extensions_. Fixed a potential zip path traversal vulnerability when unzipping Ghidra Extension archives. (GP-6354) * _Multi-User_. Upgraded yajsw to 13.18. (GP-6364) ### Bugs -* _Data Types_. Corrected Union update notification issue which impacted proper archive sync indicators and related operations. (GP-6359, Issue #8884) -* _Debugger_. Fixed missing "Dynamic Listing" entry in Window menu, when the Dynamic Listing is closed. (GP-6086, Issue #8604) -* _Debugger:Emulator_. Fixed a silent infinite read loop during some situations in an emulator forked from a live target. (GP-6340) +* _Data Types_. Corrected a Union update notification issue which impacted proper archive sync indicators and related operations. (GP-6359, Issue #8884) +* _Debugger_. Fixed missing __Dynamic Listing__ entry in Window menu, when the Dynamic Listing is closed. (GP-6086, Issue #8604) +* _Debugger:Emulator_. Fixed a silent infinite-read loop during some situations in an emulator forked from a live target. (GP-6340) * _Demangler_. Fixed Gnu Demangler failure to parse a global guard variable. (GP-6371, Issue #8900) * _GUI_. Updated the Symbol Tree's filter to fix an issue that sometimes caused it to not get painted. (GP-6366, Issue #2448) -* _Processors_. Corrected AARCH64 `ldapr` instruction semantics to properly read memory (GP-6358, Issue #6593) +* _Processors_. Corrected AARCH64 `ldapr` instruction semantics to properly read memory. (GP-6358, Issue #6593) * _Processors_. Corrected PowerPC VLE `se_blrl` instruction semantics. (GP-6379, Issue #6207) * _Processors_. Corrected issue with ARM `ldrexd` instruction when the operands are the same register. (GP-6381, Issue #6590) ### Notable API Changes * _Debugger:Emulator_. (GP-6340) Removed `PcodeTraceDataAccess.intersectUnknown` in favor of `intersectViewKnown` with sutract. -* _Emulator_. (GP-6134) Added `InstructionPrototype.hasNext2Dependency()` +* _Emulator_. (GP-6134) Added `InstructionPrototype.hasNext2Dependency()`. # Ghidra 12.0.1 Change History (January 2026) diff --git a/Ghidra/Debug/Debugger-isf/src/main/java/ghidra/dbg/isf/IsfServer.java b/Ghidra/Debug/Debugger-isf/src/main/java/ghidra/dbg/isf/IsfServer.java index 9ada15876c..f60cd30bce 100644 --- a/Ghidra/Debug/Debugger-isf/src/main/java/ghidra/dbg/isf/IsfServer.java +++ b/Ghidra/Debug/Debugger-isf/src/main/java/ghidra/dbg/isf/IsfServer.java @@ -4,9 +4,9 @@ * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at - * + * * http://www.apache.org/licenses/LICENSE-2.0 - * + * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. @@ -27,7 +27,7 @@ import ghidra.GhidraApplicationLayout; import ghidra.GhidraTestApplicationLayout; import ghidra.base.project.GhidraProject; import ghidra.framework.Application; -import ghidra.framework.GhidraApplicationConfiguration; +import ghidra.framework.HeadlessGhidraApplicationConfiguration; import ghidra.framework.data.OpenMode; import ghidra.framework.model.DomainFile; import ghidra.framework.model.ProjectData; @@ -163,8 +163,8 @@ public class IsfServer extends Thread { public static void main(String[] args) throws FileNotFoundException, IOException { GhidraApplicationLayout layout = new GhidraTestApplicationLayout(new File(AbstractGTest.getTestDirectoryPath())); - GhidraApplicationConfiguration config = new GhidraApplicationConfiguration(); - config.setShowSplashScreen(false); + HeadlessGhidraApplicationConfiguration config = + new HeadlessGhidraApplicationConfiguration(); Application.initializeApplication(layout, config); IsfServer server = new IsfServer(null, 54321); diff --git a/Ghidra/Debug/Debugger/src/main/java/ghidra/app/plugin/core/debug/gui/stack/DebuggerStackPanel.java b/Ghidra/Debug/Debugger/src/main/java/ghidra/app/plugin/core/debug/gui/stack/DebuggerStackPanel.java index 85473683ec..8689cb576d 100644 --- a/Ghidra/Debug/Debugger/src/main/java/ghidra/app/plugin/core/debug/gui/stack/DebuggerStackPanel.java +++ b/Ghidra/Debug/Debugger/src/main/java/ghidra/app/plugin/core/debug/gui/stack/DebuggerStackPanel.java @@ -73,6 +73,9 @@ public class DebuggerStackPanel extends AbstractObjectsTableBasedPanel breakpoints, ControlMode mode) { for (TraceBreakpointLocation tb : breakpoints) { try { @@ -553,7 +553,7 @@ public class DebuggerLogicalBreakpointServicePlugin extends Plugin * events that the manager punts to OBJECT_RESTORED. Thus, we have to set * forceUpdate here. */ - trackTraceBreakpoint(a, tb, mode, true); + trackTraceBreakpoint(c, tb, mode, true); } catch (TrackedTooSoonException e) { // This can still happen during reload (on OBJECT_RESTORED) @@ -579,37 +579,44 @@ public class DebuggerLogicalBreakpointServicePlugin extends Plugin new DefaultTraceLocation(trace, null, Lifespan.at(snap), minAddress)); } - protected void trackTraceBreakpoint(AddCollector a, TraceBreakpointLocation tb, - ControlMode mode, - boolean forceUpdate) throws TrackedTooSoonException { + protected void trackTraceBreakpoint(ChangeCollector c, TraceBreakpointLocation tb, + ControlMode mode, boolean forceUpdate) throws TrackedTooSoonException { if (!mode.useEmulatedBreakpoints() && (target == null || !target.isBreakpointValid(tb))) { return; } Address traceAddr = tb.getMinAddress(snap); if (traceAddr == null) { + LogicalBreakpointInternal oldLb = logicalByBreakpoint.remove(tb); + if (oldLb != null) { + // Happens when existing location's range is deleted + doRemoveFromLogicalBreakpoint(c.r, oldLb, tb); + } return; // Will update via breakpointChanged when address is set } ProgramLocation progLoc = computeStaticLocation(tb); LogicalBreakpointInternal lb; if (progLoc != null) { InfoPerProgram progInfo = programInfos.get(progLoc.getProgram()); - lb = progInfo.getOrCreateLogicalBreakpointFor(a, progLoc.getByteAddress(), tb, + lb = progInfo.getOrCreateLogicalBreakpointFor(c.a, progLoc.getByteAddress(), tb, snap); } else { - lb = getOrCreateLogicalBreakpointFor(a, traceAddr, tb, snap); + lb = getOrCreateLogicalBreakpointFor(c.a, traceAddr, tb, snap); } assert logicalByAddress.get(traceAddr).contains(lb); - logicalByBreakpoint.put(tb, lb); + LogicalBreakpointInternal oldLb = logicalByBreakpoint.put(tb, lb); + if (oldLb != lb && oldLb != null) { + // Happens when existing location's range changes + doRemoveFromLogicalBreakpoint(c.r, oldLb, tb); + } if (lb.trackBreakpoint(tb) || forceUpdate) { - a.updated(lb); + c.a.updated(lb); } } - protected LogicalBreakpointInternal removeFromLogicalBreakpoint(RemoveCollector r, - TraceBreakpointLocation tb) { - LogicalBreakpointInternal lb = logicalByBreakpoint.remove(tb); + protected LogicalBreakpointInternal doRemoveFromLogicalBreakpoint(RemoveCollector r, + LogicalBreakpointInternal lb, TraceBreakpointLocation tb) { if (lb == null || !lb.untrackBreakpoint(tb)) { return null; } @@ -625,6 +632,12 @@ public class DebuggerLogicalBreakpointServicePlugin extends Plugin return lb; } + protected LogicalBreakpointInternal removeFromLogicalBreakpoint(RemoveCollector r, + TraceBreakpointLocation tb) { + LogicalBreakpointInternal lb = logicalByBreakpoint.remove(tb); + return doRemoveFromLogicalBreakpoint(r, lb, tb); + } + protected void forgetTraceBreakpoint(RemoveCollector r, TraceBreakpointLocation tb) { LogicalBreakpointInternal lb = removeFromLogicalBreakpoint(r, tb); if (lb == null) { diff --git a/Ghidra/Debug/Framework-TraceModeling/src/main/java/ghidra/trace/database/target/DBTraceObjectInterface.java b/Ghidra/Debug/Framework-TraceModeling/src/main/java/ghidra/trace/database/target/DBTraceObjectInterface.java index 90c9107c1c..333661b048 100644 --- a/Ghidra/Debug/Framework-TraceModeling/src/main/java/ghidra/trace/database/target/DBTraceObjectInterface.java +++ b/Ghidra/Debug/Framework-TraceModeling/src/main/java/ghidra/trace/database/target/DBTraceObjectInterface.java @@ -159,6 +159,51 @@ public interface DBTraceObjectInterface extends TraceObjectInterface, TraceUniqu cast.getNewValue()); return new TraceChangeRecord<>(type, getSpace(life), iface, null, null); } + if (rec.getEventType() == TraceEvents.VALUE_LIFESPAN_CHANGED) { + if (object.isDeleted()) { + return null; + } + TraceEvent type = getChangedType(); + if (type == null) { + return null; + } + TraceChangeRecord cast = + TraceEvents.VALUE_LIFESPAN_CHANGED.cast(rec); + TraceObjectValue affected = cast.getAffectedObject(); + String key = affected.getEntryKey(); + if (!appliesToKey(key)) { + return null; + } + assert affected.getParent() == object; + if (object.getCanonicalParent(affected.getMaxSnap()) == null) { + return null; // Object is not complete + } + emitExtraValueChanged(affected.getLifespan(), key, null, null); + return new TraceChangeRecord<>(type, getSpace(life), iface, null, null); + } + if (rec.getEventType() == TraceEvents.VALUE_DELETED) { + if (object.isDeleted()) { + return null; + } + TraceEvent type = getChangedType(); + if (type == null) { + return null; + } + TraceChangeRecord cast = + TraceEvents.VALUE_DELETED.cast(rec); + TraceObjectValue affected = cast.getAffectedObject(); + String key = affected.getEntryKey(); + if (!appliesToKey(key)) { + return null; + } + assert affected.getParent() == object; + if (object.getCanonicalParent(affected.getMaxSnap()) == null) { + return null; // Object is not complete + } + emitExtraValueChanged(affected.getLifespan(), key, cast.getOldValue(), + cast.getNewValue()); + return new TraceChangeRecord<>(type, getSpace(life), iface, null, null); + } if (rec.getEventType() == TraceEvents.OBJECT_DELETED) { return translateDeleted(life); } diff --git a/Ghidra/Features/Base/src/main/help/help/topics/Annotations/Annotations.html b/Ghidra/Features/Base/src/main/help/help/topics/Annotations/Annotations.html index 5c3c9ee6c8..86974ba456 100644 --- a/Ghidra/Features/Base/src/main/help/help/topics/Annotations/Annotations.html +++ b/Ghidra/Features/Base/src/main/help/help/topics/Annotations/Annotations.html @@ -285,48 +285,6 @@ - - Execute
- - - Launches the specified executable with given optional - parameters.
- - - -
    -
  1. "executable path"
    2. -
- - OR -
    -
  1. "executable path"
    2. -
  2. "parameter list" (may be empty quotes)
    3. -
  3. "display text" (may be empty quotes)
    4. -
- - - -
    -
  • @execute
    • -
- - - -
    -
  • {@execute "C:\Program Files\Mozilla Firefox\firefox.exe"}
    • - -
  • {@execute "C:\Program Files\Mozilla Firefox\firefox.exe" - "http://my.website.com" "Opens a web browser to Website"}
    • - -
  • {@execute "C:\Program Files\Mozilla Firefox\firefox.exe" "" "My display text"}
    • - -
  • {@execute "C:\Path\To\Some\executable.exe" "arg1 arg2" ""}
    • - -
Note: quotes are required for this annotation - - - Discovered Annotations
diff --git a/Ghidra/Features/Base/src/main/help/help/topics/AutoAnalysisPlugin/AutoAnalysis.htm b/Ghidra/Features/Base/src/main/help/help/topics/AutoAnalysisPlugin/AutoAnalysis.htm index 2e40e0162c..284082788f 100644 --- a/Ghidra/Features/Base/src/main/help/help/topics/AutoAnalysisPlugin/AutoAnalysis.htm +++ b/Ghidra/Features/Base/src/main/help/help/topics/AutoAnalysisPlugin/AutoAnalysis.htm @@ -451,6 +451,13 @@ The GNU Demangler adds the following analysis options:
+

+ Timeout (seconds) - + The maximum amount of seconds to allow the native GNU Demangler process to + attempt to demangle a mangled string before failing. Some inputs to the + native GNU Demangler program have been shown to not terminate and consume excessive + resources. This timeout protects against these inputs. +

Use Deprecated Demangler - By default, GCC symbols will be demangled using the most up-to-date demangler diff --git a/Ghidra/Features/Base/src/main/java/ghidra/app/merge/datatypes/DataTypeMergeManager.java b/Ghidra/Features/Base/src/main/java/ghidra/app/merge/datatypes/DataTypeMergeManager.java index 360ce0ee97..0c3c1bf157 100644 --- a/Ghidra/Features/Base/src/main/java/ghidra/app/merge/datatypes/DataTypeMergeManager.java +++ b/Ghidra/Features/Base/src/main/java/ghidra/app/merge/datatypes/DataTypeMergeManager.java @@ -528,7 +528,7 @@ public class DataTypeMergeManager implements MergeResolver { } sb.append(dt.getDisplayName()); sb.append(", "); - if (info.index < 0) { + if (info.resultOrdinal < 0) { if (dt instanceof FunctionDefinition) { sb.append("return-type"); } @@ -539,16 +539,16 @@ public class DataTypeMergeManager implements MergeResolver { else { if (dt instanceof FunctionDefinition) { sb.append("param-"); - sb.append(info.index); + sb.append(info.resultOrdinal); } else if (dt instanceof Union) { sb.append("component-"); - sb.append(info.index); + sb.append(info.resultOrdinal); } else if (dt instanceof Structure) { Structure resultStruct = (Structure) info.ht.get(info.id); sb.append("component-"); - sb.append(info.index); + sb.append(info.resultOrdinal); if (!resultStruct.isPackingEnabled()) { sb.append(", offset-"); sb.append("0x"); @@ -556,8 +556,8 @@ public class DataTypeMergeManager implements MergeResolver { } } else { - sb.append("index-"); // unknown use case - sb.append(info.index); + sb.append("resultOrdinal-"); // unknown use case + sb.append(info.resultOrdinal); } } @@ -1254,36 +1254,19 @@ public class DataTypeMergeManager implements MergeResolver { // Add each of the defined components back in. DataTypeComponent[] comps = sourceDt.getDefinedComponents(); - // Component sequence must be flipped for components that share the same offset - // (i.e., when zero-length components exist) to ensure that the insert at offset - // does not alter the intended order. - List compList = new ArrayList<>(); - int prevOffset = -1; - int index = -1; - for (DataTypeComponent dtc : comps) { - int offset = dtc.getOffset(); - if (offset == prevOffset) { - compList.add(index, dtc); - } - else { - prevOffset = offset; - index = compList.size(); - compList.add(dtc); - } - } - comps = null; // prevent improper use - // Track dependency errors to avoid duplicate popups HashMap badIdDtMsgs = new HashMap<>(); - for (int i = 0; i < compList.size(); i++) { + for (int i = 0; i < comps.length; i++) { - DataTypeComponent sourceComp = compList.get(i); + DataTypeComponent sourceComp = comps[i]; + DataTypeComponent resultComp; DataType sourceCompDt = sourceComp.getDataType(); BitFieldDataType bfDt = null; String comment = sourceComp.getComment(); DataType resultCompDt = null; + boolean fixupRequired = false; if (sourceComp.isBitFieldComponent()) { // NOTE: primitive type will be used if unable to resolve base type @@ -1307,13 +1290,7 @@ public class DataTypeMergeManager implements MergeResolver { // Not added so should be in result if it wasn't deleted there. resultCompDt = dtms[RESULT].getDataType(sourceComponentID); } - if (resultCompDt == null) { - // Not added/resolved yet - // put an entry in the fixup list - fixUpList.add(new FixUpInfo(sourceDtID, sourceComponentID, - sourceComp.getOrdinal(), sourceComp, resolvedDataTypes)); - fixUpIDSet.add(sourceDtID); - } + fixupRequired = (resultCompDt == null); if (bfDt != null && (resultCompDt == null || !BitFieldDataType.isValidBaseDataType(resultCompDt))) { // use primitive type as fixup placeholder @@ -1335,7 +1312,8 @@ public class DataTypeMergeManager implements MergeResolver { if (packed) { if (bfDt != null) { try { - destStruct.addBitField(resultCompDt, bfDt.getDeclaredBitSize(), + resultComp = + destStruct.addBitField(resultCompDt, bfDt.getDeclaredBitSize(), sourceComp.getFieldName(), comment); } catch (InvalidDataTypeException e) { @@ -1345,7 +1323,8 @@ public class DataTypeMergeManager implements MergeResolver { comment = buildDataTypeFailureComment(sourceCompDt, e.getMessage(), sourceComp.getComment()); try { - destStruct.addBitField(primitiveBaseDt, bfDt.getDeclaredBitSize(), + resultComp = destStruct.addBitField(primitiveBaseDt, + bfDt.getDeclaredBitSize(), sourceComp.getFieldName(), comment); } catch (InvalidDataTypeException exc) { @@ -1356,13 +1335,15 @@ public class DataTypeMergeManager implements MergeResolver { else if (badMsg == null) { try { // If I have compDt, it should now be from result DTM. - destStruct.add(resultCompDt, length, sourceComp.getFieldName(), + resultComp = + destStruct.add(resultCompDt, length, sourceComp.getFieldName(), comment); } catch (IllegalArgumentException e) { comment = buildDataTypeFailureComment(sourceCompDt, e.getMessage(), comment); - destStruct.add(BadDataType.dataType, sourceComp.getLength(), + resultComp = + destStruct.add(BadDataType.dataType, sourceComp.getLength(), sourceComp.getFieldName(), comment); if (e.getCause() instanceof DataTypeDependencyException) { badIdDtMsgs.put(dtId, e.getMessage()); @@ -1373,14 +1354,15 @@ public class DataTypeMergeManager implements MergeResolver { else { // Preserve non-fixup ordinal with bad placeholder comment = buildDataTypeFailureComment(sourceCompDt, badMsg, comment); - destStruct.add(BadDataType.dataType, sourceComp.getLength(), + resultComp = destStruct.add(BadDataType.dataType, sourceComp.getLength(), sourceComp.getFieldName(), badMsg + "; " + comment); } } else if (bfDt != null) { // non-packed bitfield try { - destStruct.insertBitFieldAt(sourceComp.getOffset(), sourceComp.getLength(), + resultComp = destStruct + .insertBitFieldAt(sourceComp.getOffset(), sourceComp.getLength(), bfDt.getBitOffset(), resultCompDt, bfDt.getDeclaredBitSize(), sourceComp.getFieldName(), comment); } @@ -1391,7 +1373,8 @@ public class DataTypeMergeManager implements MergeResolver { comment = buildDataTypeFailureComment(sourceCompDt, e.getMessage(), sourceComp.getComment()); try { - destStruct.addBitField(primitiveBaseDt, bfDt.getDeclaredBitSize(), + resultComp = + destStruct.addBitField(primitiveBaseDt, bfDt.getDeclaredBitSize(), sourceComp.getFieldName(), comment); } catch (InvalidDataTypeException exc) { @@ -1404,9 +1387,9 @@ public class DataTypeMergeManager implements MergeResolver { if (badMsg == null) { try { // If not last component must constrain length to original component size - if (i < compList.size() - 1) { + if (i < (comps.length - 1)) { int offset = sourceComp.getOffset(); - DataTypeComponent nextDtc = compList.get(i + 1); + DataTypeComponent nextDtc = comps[i + 1]; int available = nextDtc.getOffset() - offset; if (length > available) { // The data type is too big, so adjust the component length to what will fit. @@ -1423,13 +1406,15 @@ public class DataTypeMergeManager implements MergeResolver { } } - destStruct.insertAtOffset(sourceComp.getOffset(), resultCompDt, length, + resultComp = destStruct.insertAtOffset(sourceComp.getOffset(), + resultCompDt, length, sourceComp.getFieldName(), comment); } catch (IllegalArgumentException e) { comment = buildDataTypeFailureComment(sourceCompDt, e.getMessage(), comment); - destStruct.insertAtOffset(sourceComp.getOffset(), BadDataType.dataType, + resultComp = destStruct.insertAtOffset(sourceComp.getOffset(), + BadDataType.dataType, length, sourceComp.getFieldName(), comment); if (e.getCause() instanceof DataTypeDependencyException) { @@ -1441,7 +1426,8 @@ public class DataTypeMergeManager implements MergeResolver { else { // Preserve non-fixup ordinal with bad placeholder comment = buildDataTypeFailureComment(sourceCompDt, badMsg, comment); - destStruct.insertAtOffset(sourceComp.getOffset(), BadDataType.dataType, + resultComp = + destStruct.insertAtOffset(sourceComp.getOffset(), BadDataType.dataType, sourceComp.getLength(), sourceComp.getFieldName(), comment); } } @@ -1450,15 +1436,22 @@ public class DataTypeMergeManager implements MergeResolver { // Add fixup placeholder to prevent the ordinal values and component sizes from // changing. Nothing we can do about packing which may be affected. // These should get fixed-up later. - destStruct.add(BadDataType.dataType, sourceComp.getLength(), + resultComp = destStruct.add(BadDataType.dataType, sourceComp.getLength(), sourceComp.getFieldName(), comment); } else { // Add fixup placeholder to prevent the ordinal values and component sizes from // changing. These should get fixed-up later. - destStruct.insertAtOffset(sourceComp.getOffset(), BadDataType.dataType, + resultComp = destStruct.insertAtOffset(sourceComp.getOffset(), BadDataType.dataType, sourceComp.getLength(), sourceComp.getFieldName(), comment); } + + if (fixupRequired) { + // Component datatype has not been added/resolved yet, put an entry in the fixup list + fixUpList.add(new FixUpInfo(sourceDtID, sourceComponentID, + resultComp.getOrdinal(), sourceComp, resolvedDataTypes)); + fixUpIDSet.add(sourceDtID); + } } if (!packed) { adjustStructureSize(destStruct, sourceDt.getLength()); @@ -2548,22 +2541,22 @@ public class DataTypeMergeManager implements MergeResolver { long lastChangeTime = fd.getLastChangeTime(); // Don't let the time change. try { if (dt != null) { - if (info.index < 0) { // -1 for return type + if (info.resultOrdinal < 0) { // -1 for return type fd.setReturnType(dt); } else { ParameterDefinition[] args = fd.getArguments(); - args[info.index].setDataType(dt); + args[info.resultOrdinal].setDataType(dt); } return true; } - if (info.index < 0) { // -1 for return type + if (info.resultOrdinal < 0) { // -1 for return type // nowhere to set error comment } else { ParameterDefinition[] args = fd.getArguments(); - ParameterDefinition arg = args[info.index]; + ParameterDefinition arg = args[info.resultOrdinal]; String comment = buildDataTypeFailureComment(info.componentDataType, null, arg.getComment()); arg.setComment(comment); @@ -2583,15 +2576,15 @@ public class DataTypeMergeManager implements MergeResolver { * @return false if component not found, else true */ private boolean fixUpPackedStructureComponent(FixUpInfo info, Structure struct, DataType dt) { - int ordinal = info.index; + int ordinal = info.resultOrdinal; - DataTypeComponent dtc = null; + DataTypeComponent dtc; if (ordinal >= 0 || ordinal < struct.getNumComponents()) { dtc = struct.getComponent(ordinal); } - - if (dtc == null) { - throw new AssertException("Expected bad datatype placeholder"); + else { + throw new AssertException( + "Expected fixup component at ordinal " + ordinal + " in " + struct.getPathName()); } long lastChangeTime = struct.getLastChangeTime(); // Don't let the time change. @@ -2680,12 +2673,16 @@ public class DataTypeMergeManager implements MergeResolver { private boolean fixUpNonPackedStructureComponent(FixUpInfo info, Structure struct, DataType dt) { - int ordinal = info.index; + int ordinal = info.resultOrdinal; - DataTypeComponent dtc = null; + DataTypeComponent dtc; if (ordinal >= 0 || ordinal < struct.getNumComponents()) { dtc = struct.getComponent(ordinal); } + else { + throw new AssertException( + "Expected fixup component at ordinal " + ordinal + " in " + struct.getPathName()); + } long lastChangeTime = struct.getLastChangeTime(); // Don't let the time change. try { @@ -2740,35 +2737,13 @@ public class DataTypeMergeManager implements MergeResolver { } // handle non-bitfield component fixup - int offset = dtc.getOffset(); - int dtcLength = dtc.getLength(); int length = dt.getLength(); if (length <= 0) { - length = dtcLength; - } - int bytesNeeded = length - dtcLength; - if (bytesNeeded > 0) { - int nextOffset = offset + dtcLength; - DataTypeComponent nextDefinedDtc = - struct.getDefinedComponentAtOrAfterOffset(nextOffset); - if (nextDefinedDtc != null) { - int bytesAvailable = nextDefinedDtc.getOffset() - nextOffset; - if (bytesAvailable < bytesNeeded) { - // The data type is too big, so adjust the component length to what will fit. - length = dtcLength + bytesAvailable; - // Output a warning indicating the structure has a data type that doesn't fit. - String message = "Structure Merge: Not enough undefined bytes to fit " + - dt.getPathName() + " in structure " + struct.getPathName() + - " at offset 0x" + Integer.toHexString(offset) + "." + - "\nIt needs " + (bytesNeeded - bytesAvailable) + - " more byte(s) to be able to fit."; - Msg.warn(this, message); - } - } + length = dtc.getLength(); } try { - struct.replaceAtOffset(offset, dt, length, dtc.getFieldName(), - dtc.getComment()); + struct.replace(ordinal, dt, length, dtc.getFieldName(), dtc.getComment()); + return true; } catch (IllegalArgumentException e) { displayError(struct, e); @@ -2804,7 +2779,7 @@ public class DataTypeMergeManager implements MergeResolver { String loc; if (struct.isPackingEnabled()) { - loc = "ordinal " + info.index; + loc = "ordinal " + info.resultOrdinal; } else { loc = "offset 0x" + Integer.toHexString(info.offset); @@ -2818,7 +2793,7 @@ public class DataTypeMergeManager implements MergeResolver { DataType compDt = resolve(info.compID, info.getDataTypeManager(), info.ht); - int ordinal = info.index; + int ordinal = info.resultOrdinal; DataTypeComponent dtc = null; if (ordinal >= 0 && ordinal <= union.getNumComponents()) { @@ -2889,7 +2864,7 @@ public class DataTypeMergeManager implements MergeResolver { } Msg.warn(this, "Union Merge: Failed to resolve data type '" + info.componentDataType.getName() + - "' at ordinal " + info.index + " in " + union.getPathName()); + "' at ordinal " + info.resultOrdinal + " in " + union.getPathName()); return false; } @@ -2913,7 +2888,7 @@ public class DataTypeMergeManager implements MergeResolver { FixUpInfo info = fixUpList.get(i); // assume info applies to union if (!fixUpUnionComponent(union, info)) { Msg.warn(this, "Union Merge: Failed to apply data type at ordinal " + - info.index + " in " + union.getPathName()); + info.resultOrdinal + " in " + union.getPathName()); unresolvedFixups.add(info); } } @@ -3447,13 +3422,18 @@ public class DataTypeMergeManager implements MergeResolver { */ private class FixUpInfo implements Comparable { - final long id; - final long compID; - final DataType componentDataType; - final int index; + final long id; // Souce datatype ID + final long compID; // Source datatype ID for component + final DataType componentDataType; // Source component datatype + + // Result ordinal when id represents a container such as a structure/union or + // function definition. In such cases, 'compID' corresponds to the component datatype. + // A -1 may be used when not applicable. + final int resultOrdinal; + final MyIdentityHashMap ht; - // component offset - for display/logging use only + // source component offset - for display/logging use only // only meaningful for non-packed structure // may not be unique (e.g., bitfields, 0-length components) int offset = -1; @@ -3468,16 +3448,16 @@ public class DataTypeMergeManager implements MergeResolver { * @param id source data type ID needing to be fixed up * @param compID source datatype ID of either param/component or bitfield base type * @param componentDataType source component/dependency datatype - * @param index offset into non-packed structure, or ordinal into union or packed - * structure; or parameter/return ordinal; for other data types index is not used (specify -1). + * @param resultOrdinal the result ordinal into a structure/union; or + * parameter/return ordinal; or -1 for other cases where index is not used * @param resolvedDataTypes hashtable used for resolving the data type */ - FixUpInfo(long id, long compID, DataType componentDataType, int index, + FixUpInfo(long id, long compID, DataType componentDataType, int resultOrdinal, MyIdentityHashMap resolvedDataTypes) { this.id = id; this.compID = compID; this.componentDataType = componentDataType; - this.index = index; + this.resultOrdinal = resultOrdinal; this.ht = resolvedDataTypes; if (componentDataType instanceof BitFieldDataType) { @@ -3491,13 +3471,13 @@ public class DataTypeMergeManager implements MergeResolver { * or components were resolved. * @param id id of data type needing to be fixed up * @param compID datatype ID of either param/component or bitfield base type - * @param destOrdinal component ordinal within destination composite + * @param resultOrdinal component ordinal within result composite * @param sourceDtc associated composite datatype component * @param resolvedDataTypes hashtable used for resolving the data type */ - FixUpInfo(long id, long compID, int destOrdinal, DataTypeComponent sourceDtc, + FixUpInfo(long id, long compID, int resultOrdinal, DataTypeComponent sourceDtc, MyIdentityHashMap resolvedDataTypes) { - this(id, compID, getDataType(sourceDtc), destOrdinal, resolvedDataTypes); + this(id, compID, getDataType(sourceDtc), resultOrdinal, resolvedDataTypes); offset = sourceDtc.getOffset(); if (sourceDtc.isBitFieldComponent()) { BitFieldDataType bfDt = (BitFieldDataType) sourceDtc.getDataType(); @@ -3516,11 +3496,11 @@ public class DataTypeMergeManager implements MergeResolver { @Override public int compareTo(FixUpInfo o) { - // Compare such that items are grouped by id and sort such that the greatest index + // Compare such that items are grouped by id and sort such that the greatest resultOrdinal // is first within that group. long c = id - o.id; if (c == 0) { - c = Integer.toUnsignedLong(o.index) - Integer.toUnsignedLong(index); + c = Integer.toUnsignedLong(o.resultOrdinal) - Integer.toUnsignedLong(resultOrdinal); } if (c == 0) { return 0; @@ -3546,7 +3526,8 @@ public class DataTypeMergeManager implements MergeResolver { } return "\n" + "ID = " + Long.toHexString(id) + ",\ndt = " + dtm.getDataType(id) + ",\ncomponent ID = " + Long.toHexString(compID) + ",\ncomponent dt = " + - dtm.getDataType(compID) + ",\noffset/index = " + index + ",\n" + bitInfo + "ht = " + + dtm.getDataType(compID) + ",\nresultOrdinal = " + resultOrdinal + ",\n" + bitInfo + + "ht = " + htStr + "\n"; } diff --git a/Ghidra/Features/Base/src/main/java/ghidra/app/util/EolComments.java b/Ghidra/Features/Base/src/main/java/ghidra/app/util/EolComments.java index 6e49c524b0..0b8075c3aa 100644 --- a/Ghidra/Features/Base/src/main/java/ghidra/app/util/EolComments.java +++ b/Ghidra/Features/Base/src/main/java/ghidra/app/util/EolComments.java @@ -40,6 +40,28 @@ import util.CollectionUtils; * Repeatable comment for the code unit, any repeatable comments for the code units that this code * unit has references to, and possibly a comment indicating the data at a code unit that is * referenced by this code unit. + * + * This section describes the various EOL comment types that may appear. The user can + * toggle which types are enabled. The comments are displayed in the order listed below. + * + * EOL Types: + * + * - EOL - user end of line comment + * + * - Repeatable - user repeatable source comment *at the code unit* + * - Ref Repeatable- for every reference *from a code unit*, show the target: + * - address repeatable, + * - function repeatable, + * - code unit repeatable + * + * - Auto - fabricated reference preview: + * - function, + * - indirect data pointer, + * - direct data access preview + * *depending on the options, this typically do not appear when + * repeatable comments exist + * + * - Offcut - comments at addresses inside of a code unit */ public class EolComments { @@ -67,6 +89,7 @@ public class EolComments { this.operandsShowReferences = operandsShowReferences; this.maxDisplayComments = maxDisplayComments; this.extraCommentsOption = extraCommentsOption; + loadComments(); } diff --git a/Ghidra/Features/Base/src/main/java/ghidra/app/util/importer/OptionChooser.java b/Ghidra/Features/Base/src/main/java/ghidra/app/util/importer/OptionChooser.java index e2de867069..5a5e8164f4 100644 --- a/Ghidra/Features/Base/src/main/java/ghidra/app/util/importer/OptionChooser.java +++ b/Ghidra/Features/Base/src/main/java/ghidra/app/util/importer/OptionChooser.java @@ -47,11 +47,10 @@ public interface OptionChooser { * Gets the {@link Loader} arguments associated with this {@link OptionChooser} * * @return The {@link Loader} arguments associated with this {@link OptionChooser} - * @throws UnsupportedOperationException if a subclass has not implemented this method * @deprecated Use {@link ProgramLoader.Builder#loaderArgs(List)} instead */ @Deprecated(since = "12.0", forRemoval = true) public default List> getArgs() { - throw new UnsupportedOperationException(); + return List.of(); } } diff --git a/Ghidra/Features/Base/src/main/java/ghidra/app/util/viewer/field/EolCommentFieldFactory.java b/Ghidra/Features/Base/src/main/java/ghidra/app/util/viewer/field/EolCommentFieldFactory.java index 1370f3b2a2..7b545eb6ec 100644 --- a/Ghidra/Features/Base/src/main/java/ghidra/app/util/viewer/field/EolCommentFieldFactory.java +++ b/Ghidra/Features/Base/src/main/java/ghidra/app/util/viewer/field/EolCommentFieldFactory.java @@ -17,8 +17,8 @@ package ghidra.app.util.viewer.field; import java.awt.Color; import java.math.BigInteger; -import java.util.ArrayList; -import java.util.List; +import java.util.*; +import java.util.function.Consumer; import docking.widgets.fieldpanel.field.*; import docking.widgets.fieldpanel.support.*; @@ -26,6 +26,7 @@ import generic.theme.GThemeDefaults.Colors.Palette; import ghidra.app.util.*; import ghidra.app.util.viewer.field.ListingColors.CommentColors; import ghidra.app.util.viewer.format.FieldFormatModel; +import ghidra.app.util.viewer.listingpanel.ListingModel; import ghidra.app.util.viewer.options.OptionsGui; import ghidra.app.util.viewer.proxy.ProxyObj; import ghidra.framework.options.*; @@ -33,8 +34,10 @@ import ghidra.program.model.address.Address; import ghidra.program.model.listing.*; import ghidra.program.util.*; import ghidra.util.HelpLocation; +import ghidra.util.StringUtilities; import ghidra.util.bean.field.AnnotatedTextFieldElement; import ghidra.util.exception.AssertException; +import utility.function.Dummy; /** * Generates End of line comment Fields. @@ -236,6 +239,7 @@ public class EolCommentFieldFactory extends FieldFactory { if (!enabled || !(obj instanceof CodeUnit)) { return null; } + CodeUnit cu = (CodeUnit) obj; Program program = cu.getProgram(); @@ -244,14 +248,8 @@ public class EolCommentFieldFactory extends FieldFactory { // comments if open. If this was allowed, then the comment would appear // on the outside data container and on the 1st internal member // - if (cu instanceof Data) { - Data data = (Data) cu; - if (data.getNumComponents() > 0) { - boolean isOpen = proxy.getListingLayoutModel().isOpen((Data) proxy.getObject()); - if (isOpen) { - return null; // avoid double showing - } - } + if (isOpenData(cu, proxy)) { + return null; } EolComments comments = new EolComments(cu, codeUnitFormatOptions.followReferencedPointers(), @@ -261,25 +259,59 @@ public class EolCommentFieldFactory extends FieldFactory { List elementList = new ArrayList<>(); AttributedString prefix = createPrefix(CommentStyle.EOL); List eols = comments.getEOLComments(); - List eolElements = convertToFieldElements(program, eols, prefix, 0); + List eolElements = convertToFieldElements(program, eols, prefix, 0, true); elementList.addAll(eolElements); + /* + This section describes the various EOL comment types that may appear. The user can + toggle which types are enabled. The comments are displayed in the order listed below. + + EOL Types: + + - EOL - user end of line comment + + - Repeatable - user repeatable source comment *at the code unit* + - Ref Repeatable- for every reference *from a code unit*, show the target: + - address repeatable, + - function repeatable, + - code unit repeatable + + - Auto - fabricated reference preview: + - function, + - indirect data pointer, + - direct data access preview + *depending on the options, this typically do not appear when + repeatable comments exist + + - Offcut - comments at addresses inside of a code unit + */ + if (comments.isShowingRepeatables()) { prefix = createPrefix(CommentStyle.REPEATABLE); int row = getNextRow(elementList); List repeatables = comments.getRepeatableComments(); - List elements = convertToFieldElements(program, repeatables, prefix, row); + List elements = + convertToFieldElements(program, repeatables, prefix, row, true); elementList.addAll(elements); } if (comments.isShowingRefRepeatables()) { - prefix = createPrefix(CommentStyle.REF_REPEATABLE); + + AttributedString refPrefix = createPrefix(CommentStyle.REF_REPEATABLE); List refRepeatables = comments.getReferencedRepeatableComments(); for (RefRepeatComment comment : refRepeatables) { + int row = getNextRow(elementList); String[] lines = comment.getCommentLines(); + Address refAddress = comment.getAddress(); + List linesList = Arrays.asList(lines); + + Consumer> decorator = elements -> { + prependRefAddress(program, refPrefix, refAddress, elements); + }; + List elements = - convertToRefFieldElements(lines, program, prefix, comment.getAddress(), row); + convertToFieldElements(program, linesList, decorator, prefix, row, true); elementList.addAll(elements); } } @@ -288,7 +320,11 @@ public class EolCommentFieldFactory extends FieldFactory { prefix = createPrefix(CommentStyle.AUTO); int row = getNextRow(elementList); List autos = comments.getAutomaticComment(); - List elements = convertToFieldElements(program, autos, prefix, row); + + // Note: we pass 'false' for allowing annotations so that the user will see the raw data + // and not an interpreted annotation. + List elements = + convertToFieldElements(program, autos, prefix, row, false); elementList.addAll(elements); } @@ -296,7 +332,8 @@ public class EolCommentFieldFactory extends FieldFactory { prefix = createPrefix(CommentStyle.OFFCUT); int row = getNextRow(elementList); List offcuts = comments.getOffcutEolComments(); - List elements = convertToFieldElements(program, offcuts, prefix, row); + List elements = + convertToFieldElements(program, offcuts, prefix, row, true); elementList.addAll(elements); } @@ -307,6 +344,18 @@ public class EolCommentFieldFactory extends FieldFactory { maxDisplayLines, hlProvider); } + private boolean isOpenData(CodeUnit cu, ProxyObj proxy) { + if (cu instanceof Data data) { + if (data.getNumComponents() > 0) { + ListingModel listingModel = proxy.getListingLayoutModel(); + if (listingModel.isOpen(data)) { + return true; + } + } + } + return false; + } + private AttributedString createPrefix(CommentStyle commentStyle) { if (commentStyle == CommentStyle.EOL) { return new AttributedString(SEMICOLON_PREFIX, CommentColors.EOL, getMetrics(style), @@ -349,7 +398,15 @@ public class EolCommentFieldFactory extends FieldFactory { } private List convertToFieldElements(Program program, List comments, - AttributedString prefix, int row) { + AttributedString prefix, int row, boolean allowAnnotations) { + + Consumer> decorator = Dummy.consumer(); // no decorations by default + return convertToFieldElements(program, comments, decorator, prefix, row, allowAnnotations); + } + + private List convertToFieldElements(Program program, List comments, + Consumer> decorator, AttributedString prefix, int row, + boolean allowAnnotations) { List fieldElements = new ArrayList<>(); if (comments.isEmpty()) { @@ -357,11 +414,15 @@ public class EolCommentFieldFactory extends FieldFactory { } for (int commentRow = 0; commentRow < comments.size(); commentRow++) { - int offsetRow = row + commentRow; - fieldElements.add(CommentUtils.parseTextForAnnotations(comments.get(commentRow), - program, prefix, offsetRow)); + int encodedRow = row + commentRow; + String commentText = comments.get(commentRow); + FieldElement element = + createCommentField(commentText, prefix, program, encodedRow, allowAnnotations); + fieldElements.add(element); } + decorator.accept(fieldElements); + if (isWordWrap) { int lineWidth = showSemicolon ? width - prefix.getStringWidth() : width; fieldElements = FieldUtils.wrap(fieldElements, lineWidth); @@ -381,57 +442,43 @@ public class EolCommentFieldFactory extends FieldFactory { return fieldElements; } - private List convertToRefFieldElements(String[] comments, Program program, - AttributedString currentPrefixString, Address refAddress, int nextRow) { + private FieldElement createCommentField(String commentText, AttributedString prototype, + Program program, int row, boolean allowAnnotations) { - int numCommentLines = comments.length; - List fieldElements = new ArrayList<>(); - if (numCommentLines == 0) { - return fieldElements; - } - for (int rowIndex = 0; rowIndex < numCommentLines; rowIndex++) { - int encodedRow = nextRow + rowIndex; - fieldElements.add(CommentUtils.parseTextForAnnotations(comments[rowIndex], program, - currentPrefixString, encodedRow)); - } - if (prependRefAddress) { - FieldElement commentElement = fieldElements.get(0); - // Address - String refAddrComment = "{@address " + refAddress.toString() + "}"; - RowColLocation startRowCol = commentElement.getDataLocationForCharacterIndex(0); - int encodedRow = startRowCol.row(); - int encodedCol = startRowCol.col(); - Annotation annotation = new Annotation(refAddrComment, program); - FieldElement addressElement = - new AnnotatedTextFieldElement(annotation, currentPrefixString, program, encodedRow, - encodedCol); - - // Space character - AttributedString spaceStr = new AttributedString(" ", currentPrefixString.getColor(0), - currentPrefixString.getFontMetrics(0), false, null); - FieldElement spacerElement = new TextFieldElement(spaceStr, encodedRow, encodedCol); - fieldElements.add(new CompositeFieldElement( - new FieldElement[] { addressElement, spacerElement, commentElement })); + if (allowAnnotations) { + return CommentUtils.parseTextForAnnotations(commentText, program, prototype, row); } - if (isWordWrap) { - int lineWidth = showSemicolon ? width - currentPrefixString.getStringWidth() : width; - fieldElements = FieldUtils.wrap(fieldElements, lineWidth); + String text = StringUtilities.convertTabsToSpaces(commentText); + AttributedString as = new AttributedString(text, prototype.getColor(0), + prototype.getFontMetrics(0), false, null); + return new TextFieldElement(as, row, 0); + } + + private void prependRefAddress(Program program, AttributedString prefix, Address refAddress, + List fieldElements) { + + if (!prependRefAddress) { + return; } - if (showSemicolon) { - for (int i = 0; i < fieldElements.size(); i++) { - RowColLocation startRowCol = - fieldElements.get(i).getDataLocationForCharacterIndex(0); - int encodedRow = startRowCol.row(); - int encodedCol = startRowCol.col(); - FieldElement prefixFieldElement = - new TextFieldElement(currentPrefixString, encodedRow, encodedCol); - fieldElements.set(i, new CompositeFieldElement( - new FieldElement[] { prefixFieldElement, fieldElements.get(i) })); - } - } - return fieldElements; + FieldElement commentElement = fieldElements.get(0); + + // Address + String refAddrComment = "{@address " + refAddress.toString() + "}"; + RowColLocation startRowCol = commentElement.getDataLocationForCharacterIndex(0); + int encodedRow = startRowCol.row(); + int encodedCol = startRowCol.col(); + Annotation annotation = new Annotation(refAddrComment, program); + FieldElement addressElement = + new AnnotatedTextFieldElement(annotation, prefix, program, encodedRow, encodedCol); + + // Space character + AttributedString spaceStr = new AttributedString(" ", prefix.getColor(0), + prefix.getFontMetrics(0), false, null); + FieldElement spacerElement = new TextFieldElement(spaceStr, encodedRow, encodedCol); + fieldElements.set(0, new CompositeFieldElement( + new FieldElement[] { addressElement, spacerElement, commentElement })); } /** diff --git a/Ghidra/Features/Base/src/main/java/ghidra/app/util/viewer/field/ExecutableTaskStringHandler.java b/Ghidra/Features/Base/src/main/java/ghidra/app/util/viewer/field/ExecutableTaskStringHandler.java deleted file mode 100644 index 2952bf5b42..0000000000 --- a/Ghidra/Features/Base/src/main/java/ghidra/app/util/viewer/field/ExecutableTaskStringHandler.java +++ /dev/null @@ -1,198 +0,0 @@ -/* ### - * IP: GHIDRA - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package ghidra.app.util.viewer.field; - -import java.io.*; -import java.util.*; - -import docking.widgets.fieldpanel.field.AttributedString; -import ghidra.app.nav.Navigatable; -import ghidra.framework.plugintool.ServiceProvider; -import ghidra.program.model.listing.Program; -import ghidra.util.Msg; - -public class ExecutableTaskStringHandler implements AnnotatedStringHandler { - private static final String INVALID_SYMBOL_TEXT = - "@execute annotation must have an executable name"; - private static final String[] SUPPORTED_ANNOTATIONS = { "execute" }; - - @Override - public AttributedString createAnnotatedString(AttributedString prototypeString, String[] text, - Program program) throws AnnotationException { - if (text.length <= 1) { - throw new AnnotationException(INVALID_SYMBOL_TEXT); - } - - String displayText = getDisplayText(text); - if (displayText == null) { - // some kind of error - throw new AnnotationException(INVALID_SYMBOL_TEXT); - } - - return new AttributedString(displayText, prototypeString.getColor(0), - prototypeString.getFontMetrics(0), true, prototypeString.getColor(0)); - } - - private String getDisplayText(String[] text) { - // - // We currently support two modes of: 3 parameters or 1. The user can leave off the - // executable's parameter and display string OR they can have all three. - // - if (text.length == 4) { - return text[3]; // 4 items means they have display text - } - else if (text.length != 2) { - throw new AnnotationException( - "Invalid number of inputs - " + (text.length - 1) + " found - 1 or 3 required"); - } - - // otherwise, no display text, just use the executable name - String programInfo = text[1]; - return getDisplayTextForFilePathOrName(programInfo); - } - - private String getDisplayTextForFilePathOrName(String fileString) { - File file = new File(fileString); - if (file.isAbsolute() && file.exists()) { - return file.getName(); - } - return fileString; - } - - @Override - public String getDisplayString() { - return "Execute"; - } - - @Override - public String getPrototypeString() { - return "{@execute \"executable_path_and_name\" \"arg1 arg2\" \"Display Text\"}"; - } - - @Override - public String[] getSupportedAnnotations() { - return SUPPORTED_ANNOTATIONS; - } - - @Override - public boolean handleMouseClick(String[] annotationParts, Navigatable sourceNavigatable, - ServiceProvider serviceProvider) { - - String executableName = annotationParts[1]; - - List command = new ArrayList<>(); - command.add(executableName); - - if (annotationParts.length > 2) { - String commandParameterString = annotationParts[2]; - StringTokenizer tokenizer = new StringTokenizer(commandParameterString, " "); - while (tokenizer.hasMoreTokens()) { - command.add(tokenizer.nextToken()); - } - } - - new ProcessThread(command).start(); - - return true; - } - -//================================================================================================== -// Inner Classes -//================================================================================================== - - private static class ProcessThread extends Thread { - - private final List command; - - ProcessThread(List command) { - super("Process Runner - " + command.get(0)); - this.command = command; - } - - @Override - public void run() { - ProcessBuilder processBuilder = new ProcessBuilder(command); - processBuilder = processBuilder.redirectErrorStream(true); - - IOThread ioThread = null; - StringBuilder buffer = new StringBuilder(); - int exitValue = 1; - InputStream inputStream = null; - Process process = null; - String executableName = command.get(0); - try { - Msg.info(this, "Launching process: " + executableName); - process = processBuilder.start(); - inputStream = process.getInputStream(); - ioThread = new IOThread(buffer, inputStream); - ioThread.start(); - exitValue = process.waitFor(); - ioThread.join(); - inputStream.close(); - } - catch (Exception e) { - Msg.showError(this, null, "Error Launching Executable", - "Unexpected exception trying to launch process: " + executableName, e); - - } - finally { - if (inputStream != null) { - try { - inputStream.close(); - } - catch (IOException e) { - // ignore; we tried - } - } - } - - if (exitValue != 0) { - Msg.warn(this, "Process \"" + executableName + "\" exited abnormally with value: " + - exitValue); - } - } - } - - private static class IOThread extends Thread { - private BufferedReader shellOutput; - private StringBuilder buffer; - - IOThread(StringBuilder buffer, InputStream input) { - super("IO Thread - Executable Annotation Task"); - this.buffer = buffer; - shellOutput = new BufferedReader(new InputStreamReader(input)); - } - - @Override - public void run() { - String line = null; - try { - while ((line = shellOutput.readLine()) != null) { - buffer.append(line).append('\n'); - } - } - catch (Exception e) { - Msg.error(this, "Exception reading output for executable annotation", e); - buffer = null; - } - } - } - - @Override - public String getPrototypeString(String displayText) { - return "{@execute " + displayText.trim() + "}"; - } -} diff --git a/Ghidra/Features/Base/src/main/java/ghidra/framework/GhidraApplicationConfiguration.java b/Ghidra/Features/Base/src/main/java/ghidra/framework/GhidraApplicationConfiguration.java index 3f0c43c8ce..6ee372a5ec 100644 --- a/Ghidra/Features/Base/src/main/java/ghidra/framework/GhidraApplicationConfiguration.java +++ b/Ghidra/Features/Base/src/main/java/ghidra/framework/GhidraApplicationConfiguration.java @@ -4,9 +4,9 @@ * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at - * + * * http://www.apache.org/licenses/LICENSE-2.0 - * + * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. @@ -15,6 +15,8 @@ */ package ghidra.framework; +import java.awt.GraphicsEnvironment; + import docking.DockingErrorDisplay; import docking.DockingWindowManager; import docking.framework.ApplicationInformationDisplayFactory; @@ -37,6 +39,16 @@ public class GhidraApplicationConfiguration extends HeadlessGhidraApplicationCon private static final String USER_AGREEMENT_PROPERTY_NAME = "USER_AGREEMENT"; private boolean showSplashScreen = true; + public GhidraApplicationConfiguration() { + if (GraphicsEnvironment.isHeadless()) { + System.err.println( + "ERROR: Unable to launch Ghidra GUI application in headless environment."); + System.err.println( + "If launching from a remote SSH shell, you must have an X11 compatible client with X11 forwarding enabled."); + System.exit(1); + } + } + @Override public boolean isHeadless() { return false; diff --git a/Ghidra/Features/Base/src/main/java/ghidra/program/database/ProgramBuilder.java b/Ghidra/Features/Base/src/main/java/ghidra/program/database/ProgramBuilder.java index 23add48ab5..d14cf1f851 100644 --- a/Ghidra/Features/Base/src/main/java/ghidra/program/database/ProgramBuilder.java +++ b/Ghidra/Features/Base/src/main/java/ghidra/program/database/ProgramBuilder.java @@ -866,6 +866,18 @@ public class ProgramBuilder { } } + /** + * Creates a non-null-terminated ascii string at the given address + * @param address the address + * @param string the string + * @return the new data + * @throws Exception if there is an exception + */ + public Data createString(String address, String string) throws Exception { + return createString(address, string, StandardCharsets.US_ASCII, false, + StringDataType.dataType); + } + public Data createString(String address, String string, Charset charset, boolean nullTerminate, DataType dataType) throws Exception { if (nullTerminate) { diff --git a/Ghidra/Features/Base/src/test.slow/java/ghidra/app/merge/datatypes/DataTypeMerge8Test.java b/Ghidra/Features/Base/src/test.slow/java/ghidra/app/merge/datatypes/DataTypeMerge8Test.java index 73375c6f91..4ca35f2e23 100644 --- a/Ghidra/Features/Base/src/test.slow/java/ghidra/app/merge/datatypes/DataTypeMerge8Test.java +++ b/Ghidra/Features/Base/src/test.slow/java/ghidra/app/merge/datatypes/DataTypeMerge8Test.java @@ -115,6 +115,8 @@ public class DataTypeMerge8Test extends AbstractDataTypeMergeTest { // choose MY for Bar conflict chooseOption(DataTypeMergeManager.OPTION_MY); + pressButtonByName(waitForWindow("Structure Update Failed"), "OK"); // expected dependency error on ABC + waitForCompletion(); FrontEndPlugin frontEndPlugin = getPlugin(frontEndTool, FrontEndPlugin.class); @@ -122,8 +124,9 @@ public class DataTypeMerge8Test extends AbstractDataTypeMergeTest { JLabel label = (JLabel) TestUtils.getInstanceField("label", logPanel); String statusText = label.getText(); String expectedText = - "Structure Merge: Not enough undefined bytes to fit /XYZ in structure " + - "/MISC/ABC at offset 0x4. It needs 3 more byte(s) to be able to fit."; + "Structure Update Failed: Some of your changes to ABC cannot be merged. " + + "Problem: Not enough undefined bytes to fit /XYZ in structure /MISC/ABC at " + + "offset 0x4. It needs 3 more byte(s) to be able to fit."; assertTrue("Wrong status text: " + statusText, statusText.contains(expectedText)); } } diff --git a/Ghidra/Features/Base/src/test.slow/java/ghidra/app/merge/datatypes/DataTypeMergeFixupTest.java b/Ghidra/Features/Base/src/test.slow/java/ghidra/app/merge/datatypes/DataTypeMergeFixupTest.java index 12a0cae1d3..99049264df 100644 --- a/Ghidra/Features/Base/src/test.slow/java/ghidra/app/merge/datatypes/DataTypeMergeFixupTest.java +++ b/Ghidra/Features/Base/src/test.slow/java/ghidra/app/merge/datatypes/DataTypeMergeFixupTest.java @@ -367,4 +367,85 @@ public class DataTypeMergeFixupTest extends AbstractDataTypeMergeTest { //@formatter:on } + @Test + public void testNonPackedZeroLengthComponentFixup() throws Exception { + + // Goal is to fixup zero-length component at end of structure where its ordinal will + // be revised during the merge processing + + final CategoryPath rootPath = new CategoryPath("/"); + + mtf.initialize("notepad", new OriginalProgramModifierListener() { + + @Override + public void modifyOriginal(ProgramDB program) throws Exception { + DataTypeManager dtm = program.getDataTypeManager(); + + Union inner = new UnionDataType("inner"); + inner.add(DWordDataType.dataType); + inner = (Union) dtm.addDataType(inner, null); + + Structure other = new StructureDataType("other", 0); + other.add(WordDataType.dataType); + other = (Structure) dtm.addDataType(other, null); + + Structure outer = new StructureDataType("outer", 20, dtm); + outer.replaceAtOffset(0, other, -1, null, null); // prevent size change + outer = (Structure) dtm.addDataType(outer, null); + assertEquals(20, outer.getLength()); + } + + @Override + public void modifyLatest(ProgramDB program) throws Exception { + DataTypeManager dtm = program.getDataTypeManager(); + + // Increase size of other struct + Structure other = (Structure) dtm.getDataType(rootPath, "other"); + other.add(DWordDataType.dataType); + + // remove inner to trigger conflict with its modification + Union inner = (Union) dtm.getDataType(rootPath, "inner"); + dtm.remove(inner); + } + + @Override + public void modifyPrivate(ProgramDB program) throws Exception { + DataTypeManager dtm = program.getDataTypeManager(); + Structure outer = (Structure) dtm.getDataType(rootPath, "outer"); + Union inner = (Union) dtm.getDataType(rootPath, "inner"); + + // change inner to trigger conflict with its removal + inner.add(WordDataType.dataType); + + // Add zero-length array at end of struct + outer.insertAtOffset(20, new ArrayDataType(inner, 0), -1); + assertEquals(20, outer.getLength()); + } + }); + + executeMerge(); + + chooseOption(DataTypeMergeManager.OPTION_MY); // resolve inner conflict + + chooseOption(DataTypeMergeManager.OPTION_MY); // resolve outer conflict + + waitForCompletion(); + + DataTypeManager dtm = resultProgram.getDataTypeManager(); + + StructureInternal outer = (StructureInternal) dtm.getDataType(rootPath, "outer"); + assertNotNull(outer); + + //@formatter:off + assertEquals("/outer\n" + + "pack(disabled)\n" + + "Structure outer {\n" + + " 0 other 6 \"\"\n" + + " 20 inner[0] 0 \"\"\n" + + "}\n" + + "Length: 20 Alignment: 1\n", outer.toString()); + //@formatter:on + + } + } diff --git a/Ghidra/Features/Base/src/test.slow/java/ghidra/app/plugin/core/codebrowser/CodeBrowserOptionsTest.java b/Ghidra/Features/Base/src/test.slow/java/ghidra/app/plugin/core/codebrowser/CodeBrowserOptionsTest.java index 597a4892ba..35a0ff822c 100644 --- a/Ghidra/Features/Base/src/test.slow/java/ghidra/app/plugin/core/codebrowser/CodeBrowserOptionsTest.java +++ b/Ghidra/Features/Base/src/test.slow/java/ghidra/app/plugin/core/codebrowser/CodeBrowserOptionsTest.java @@ -692,16 +692,16 @@ public class CodeBrowserOptionsTest extends AbstractGhidraHeadedIntegrationTest options.setBoolean(WORD_WRAP, false); cb.updateNow(); btf = (ListingTextField) cb.getCurrentField(); - assertEquals(12, getNumberOfLines(btf)); + assertEquals(11, getNumberOfLines(btf)); assertTrue("; ".equals(btf.getFieldElement(5, 0).getText())); options.setBoolean(SHOW_SEMICOLON, false); cb.updateNow(); btf = (ListingTextField) cb.getCurrentField(); - assertEquals(12, getNumberOfLines(btf)); - assertFalse("; ".equals(btf.getFieldElement(1, 0).getText())); - assertEquals("01003fa1", btf.getFieldElement(11, 4).getText()); - assertEquals("Mem ref line1.", btf.getFieldElement(11, 11).getText()); + assertEquals(11, getNumberOfLines(btf)); + assertTrue("".equals(btf.getFieldElement(1, 0).getText())); // blank line - leading ';' not present + assertEquals("01003fa1", btf.getFieldElement(9, 4).getText()); + assertEquals("Mem ref line1.", btf.getFieldElement(9, 11).getText()); options.setBoolean(SHOW_REF_ADDR, false); cb.updateNow(); diff --git a/Ghidra/Features/Base/src/test.slow/java/ghidra/app/util/viewer/field/EolCommentFieldFactoryTest.java b/Ghidra/Features/Base/src/test.slow/java/ghidra/app/util/viewer/field/EolCommentFieldFactoryTest.java index 0bd1e37717..3818e4f7be 100644 --- a/Ghidra/Features/Base/src/test.slow/java/ghidra/app/util/viewer/field/EolCommentFieldFactoryTest.java +++ b/Ghidra/Features/Base/src/test.slow/java/ghidra/app/util/viewer/field/EolCommentFieldFactoryTest.java @@ -15,6 +15,8 @@ */ package ghidra.app.util.viewer.field; +import static org.hamcrest.MatcherAssert.assertThat; +import static org.hamcrest.core.StringContains.*; import static org.hamcrest.core.StringStartsWith.*; import static org.junit.Assert.*; @@ -29,10 +31,17 @@ import ghidra.program.database.ProgramDB; import ghidra.program.model.address.Address; import ghidra.program.model.address.AddressFactory; import ghidra.program.model.listing.*; +import ghidra.program.model.symbol.RefType; +import ghidra.program.model.symbol.SourceType; import ghidra.test.*; public class EolCommentFieldFactoryTest extends AbstractGhidraHeadedIntegrationTest { + private static final String STRING_ADDRESS_WITH_ANNOTATION = "0x01002c98"; + private static final String ADDRESS_CALLING_STRING_WITH_ANNOTATION = "0X01002d37"; + private static final String AUTO_COMMENT_TEXT_WITH_ANNOTATION = + "Annotation: {@address 12345678 foo}"; + private TestEnv env; private CodeBrowserPlugin cb; private Options fieldOptions; @@ -75,21 +84,45 @@ public class EolCommentFieldFactoryTest extends AbstractGhidraHeadedIntegrationT public void testRepeatableComment_FunctionCall() throws Exception { // check existing auto comment - ListingTextField tf = getFieldText(addr("0x010022e6")); + String from = "0x010022e6"; + ListingTextField tf = getFieldText(addr(from)); assertEquals(1, tf.getNumRows()); assertThat(tf.getText(), startsWith("undefined ghidra(undefined4 param_1,")); - // set repeatable comment at destination - Address destination = addr("0x01002cf5"); + // set repeatable comment at source + String to = "0x01002cf5"; String repeatableComment = "My repeatable comment"; - setRepeatableComment(destination, repeatableComment); + setRepeatableComment(addr(to), repeatableComment); - // check that the auto comment now matches the updated comment - tf = getFieldText(addr("0x010022e6")); + // check that the repeatable comment now matches the updated comment + tf = getFieldText(addr(from)); assertEquals(1, tf.getNumRows()); assertEquals(tf.getText(), repeatableComment); } + @Test + public void testRepeatableComment_FunctionCall_PrependRefAddress() throws Exception { + + setBooleanOption(EolCommentFieldFactory.ENABLE_PREPEND_REF_ADDRESS_KEY, true); + + // check existing auto comment + String from = "0x010022e6"; + ListingTextField tf = getFieldText(addr(from)); + assertEquals(1, tf.getNumRows()); + assertThat(tf.getText(), startsWith("undefined ghidra(undefined4 param_1,")); + + // set repeatable comment at source + String to = "0x01002cf5"; + String repeatableComment = "My repeatable comment"; + setRepeatableComment(addr(to), repeatableComment); + + // check that the repeatable comment now matches the updated comment and has the ref address + // prepended + tf = getFieldText(addr(from)); + assertEquals(1, tf.getNumRows()); + assertEquals("01002cf5 " + repeatableComment, tf.getText()); + } + @Test public void testRepeatableComment_DataAccess() throws Exception { @@ -106,7 +139,21 @@ public class EolCommentFieldFactoryTest extends AbstractGhidraHeadedIntegrationT // check that the auto comment now matches the updated comment tf = getFieldText(addr("0x01002265")); assertEquals(1, tf.getNumRows()); - assertEquals(tf.getText(), repeatableComment); + assertEquals(repeatableComment, tf.getText()); + } + + @Test + public void testAutoCommentDoesNotRenderAnnotation() { + + /* + Creates a data reference to a string containing an annotation. Tests that the + annotation is not rendered, but is shown in its raw form. + */ + + goTo(env.getTool(), program, STRING_ADDRESS_WITH_ANNOTATION); + ListingTextField tf = getFieldText(addr(ADDRESS_CALLING_STRING_WITH_ANNOTATION)); + assertEquals(1, tf.getNumRows()); + assertThat(tf.getText(), containsString(AUTO_COMMENT_TEXT_WITH_ANNOTATION)); } //================================================================================================== @@ -115,6 +162,13 @@ public class EolCommentFieldFactoryTest extends AbstractGhidraHeadedIntegrationT private ProgramDB buildProgram() throws Exception { ClassicSampleX86ProgramBuilder builder = new ClassicSampleX86ProgramBuilder(); + + builder.createString(STRING_ADDRESS_WITH_ANNOTATION, AUTO_COMMENT_TEXT_WITH_ANNOTATION); + + builder.createMemoryReference(ADDRESS_CALLING_STRING_WITH_ANNOTATION, + STRING_ADDRESS_WITH_ANNOTATION, + RefType.DATA, SourceType.ANALYSIS); + return builder.getProgram(); } diff --git a/Ghidra/Features/Base/src/test/java/ghidra/program/model/data/StructureDataTypeTest.java b/Ghidra/Features/Base/src/test/java/ghidra/program/model/data/StructureDataTypeTest.java index 683aba0566..9416bde01a 100644 --- a/Ghidra/Features/Base/src/test/java/ghidra/program/model/data/StructureDataTypeTest.java +++ b/Ghidra/Features/Base/src/test/java/ghidra/program/model/data/StructureDataTypeTest.java @@ -338,6 +338,74 @@ public class StructureDataTypeTest extends AbstractGenericTest { assertEquals(ByteDataType.class, comps[2].getDataType().getClass()); } + @Test + public void testInsertAtSameOffset1() { + + DataType zeroDt = new ArrayDataType(ByteDataType.dataType, 0); + + struct = createStructure("Test", 100); + assertFalse(struct.isPackingEnabled()); + + struct.insertAtOffset(0, zeroDt, -1, "a", "comment a"); + struct.insertAtOffset(0, zeroDt, -1, "b", "comment b"); + struct.insertAtOffset(0, WordDataType.dataType, -1, "c", "comment c"); + + DataTypeComponent[] definedComponents = struct.getDefinedComponents(); + assertEquals("a", definedComponents[0].getFieldName()); + assertEquals("b", definedComponents[1].getFieldName()); + assertEquals("c", definedComponents[2].getFieldName()); + } + + @Test + public void testInsertAtSameOffset2() { + + DataType zeroDt = new ArrayDataType(ByteDataType.dataType, 0); + + struct = createStructure("Test", 100); + assertFalse(struct.isPackingEnabled()); + + struct.insertAtOffset(0, WordDataType.dataType, -1, "c", "comment c"); + struct.insertAtOffset(0, zeroDt, -1, "a", "comment a"); + struct.insertAtOffset(0, zeroDt, -1, "b", "comment b"); + + DataTypeComponent[] definedComponents = struct.getDefinedComponents(); + assertEquals("a", definedComponents[0].getFieldName()); + assertEquals("b", definedComponents[1].getFieldName()); + assertEquals("c", definedComponents[2].getFieldName()); + } + + @Test + public void testInsertAtEndOffset() { + + DataType zeroDt = new ArrayDataType(ByteDataType.dataType, 0); + + struct = createStructure("Test2", 100); + assertFalse(struct.isPackingEnabled()); + + struct.insertAtOffset(100, zeroDt, -1, "a", "comment a"); + struct.insertAtOffset(100, zeroDt, -1, "b", "comment b"); + + assertEquals("/Test2\n" + + "pack(disabled)\n" + + "Structure Test2 {\n" + + " 100 byte[0] 0 a \"comment a\"\n" + + " 100 byte[0] 0 b \"comment b\"\n" + + "}\n" + + "Length: 100 Alignment: 1\n", struct.toString()); + + // Insert non-zero-length component will increase struct size + struct.insertAtOffset(100, WordDataType.dataType, -1, "c", "comment c"); + + assertEquals("/Test2\n" + + "pack(disabled)\n" + + "Structure Test2 {\n" + + " 100 byte[0] 0 a \"comment a\"\n" + + " 100 byte[0] 0 b \"comment b\"\n" + + " 100 word 2 c \"comment c\"\n" + + "}\n" + + "Length: 102 Alignment: 1\n", struct.toString()); + } + // test inserting at offset 0 @Test public void testInsertAtOffset() { @@ -458,32 +526,18 @@ public class StructureDataTypeTest extends AbstractGenericTest { Array zeroArray = new ArrayDataType(FloatDataType.dataType, 0, -1); struct.insertAtOffset(2, zeroArray, -1); struct.insertAtOffset(2, FloatDataType.dataType, -1); - assertEquals(13, struct.getLength()); - - DataTypeComponent[] comps = struct.getDefinedComponents(); - - assertEquals(6, comps.length); - - assertEquals(0, comps[0].getOffset()); - assertEquals(0, comps[0].getOrdinal()); - assertEquals(ByteDataType.class, comps[0].getDataType().getClass()); - - assertEquals(2, comps[1].getOffset()); - assertEquals(2, comps[1].getOrdinal()); - assertEquals(FloatDataType.class, comps[1].getDataType().getClass()); - - assertEquals(6, comps[2].getOffset()); - assertEquals(3, comps[2].getOrdinal()); - assertTrue(zeroArray.isEquivalent(comps[2].getDataType())); - - assertEquals(6, comps[3].getOffset()); - assertEquals(4, comps[3].getOrdinal()); - assertEquals(WordDataType.class, comps[3].getDataType().getClass()); - - assertEquals(8, comps[4].getOffset()); - assertEquals(5, comps[4].getOrdinal()); - assertEquals(DWordDataType.class, comps[4].getDataType().getClass()); + assertEquals("/TestStruct\n" + + "pack(disabled)\n" + + "Structure TestStruct {\n" + + " 0 byte 1 field1 \"Comment1\"\n" + + " 2 float[0] 0 \"\"\n" + + " 2 float 4 \"\"\n" + + " 6 word 2 \"Comment2\"\n" + + " 8 dword 4 field3 \"\"\n" + + " 12 byte 1 field4 \"Comment4\"\n" + + "}\n" + + "Length: 13 Alignment: 1\n", struct.toString()); } @Test @@ -660,7 +714,15 @@ public class StructureDataTypeTest extends AbstractGenericTest { } @Test - public void testInsertBitFieldAtLittleEndian() throws Exception { + public void testInsertBitFieldAtLittleEndian2() throws Exception { + + Array zeroArray = new ArrayDataType(CharDataType.dataType, 0, -1); + struct.insertAtOffset(2, zeroArray, -1, "A", null); + struct.insertAtOffset(2, zeroArray, -1, "B", null); + struct.insertAtOffset(2, zeroArray, -1, "C", null); + struct.insertAtOffset(2, zeroArray, -1, "D", null); + struct.insertAtOffset(2, zeroArray, -1, "E", null); + struct.insertAtOffset(2, CharDataType.dataType, -1, "XXX", null); struct.insertBitFieldAt(2, 4, 0, IntegerDataType.dataType, 3, "bf1", "bf1Comment"); @@ -670,15 +732,21 @@ public class StructureDataTypeTest extends AbstractGenericTest { "Structure TestStruct {\n" + " 0 byte 1 field1 \"Comment1\"\n" + // " 1 undefined 1 \"\"\n" + + " 2 char[0] 0 A \"\"\n" + + " 2 char[0] 0 B \"\"\n" + + " 2 char[0] 0 C \"\"\n" + + " 2 char[0] 0 D \"\"\n" + + " 2 char[0] 0 E \"\"\n" + " 2 int:3(0) 1 bf1 \"bf1Comment\"\n" + // " 3 undefined 1 \"\"\n" + // " 4 undefined 1 \"\"\n" + -// " 5 undefined 1 \"\"\n" + - " 6 word 2 \"Comment2\"\n" + - " 8 dword 4 field3 \"\"\n" + - " 12 byte 1 field4 \"Comment4\"\n" + +// " 5 undefined 1 \"\"\n" + + " 6 char 1 XXX \"\"\n" + + " 7 word 2 \"Comment2\"\n" + + " 9 dword 4 field3 \"\"\n" + + " 13 byte 1 field4 \"Comment4\"\n" + "}\n" + - "Length: 13 Alignment: 1", struct); + "Length: 14 Alignment: 1", struct); //@formatter:on struct.insertBitFieldAt(2, 4, 3, IntegerDataType.dataType, 3, "bf2", "bf2Comment"); @@ -688,17 +756,23 @@ public class StructureDataTypeTest extends AbstractGenericTest { "pack(disabled)\n" + "Structure TestStruct {\n" + " 0 byte 1 field1 \"Comment1\"\n" + -// " 1 undefined 1 \"\"\n" + +// " 1 undefined 1 \"\"\n" + + " 2 char[0] 0 A \"\"\n" + + " 2 char[0] 0 B \"\"\n" + + " 2 char[0] 0 C \"\"\n" + + " 2 char[0] 0 D \"\"\n" + + " 2 char[0] 0 E \"\"\n" + " 2 int:3(0) 1 bf1 \"bf1Comment\"\n" + " 2 int:3(3) 1 bf2 \"bf2Comment\"\n" + // " 3 undefined 1 \"\"\n" + // " 4 undefined 1 \"\"\n" + // " 5 undefined 1 \"\"\n" + - " 6 word 2 \"Comment2\"\n" + - " 8 dword 4 field3 \"\"\n" + - " 12 byte 1 field4 \"Comment4\"\n" + + " 6 char 1 XXX \"\"\n" + + " 7 word 2 \"Comment2\"\n" + + " 9 dword 4 field3 \"\"\n" + + " 13 byte 1 field4 \"Comment4\"\n" + "}\n" + - "Length: 13 Alignment: 1", struct); + "Length: 14 Alignment: 1", struct); //@formatter:on struct.insertBitFieldAt(2, 4, 6, IntegerDataType.dataType, 15, "bf3", "bf3Comment"); @@ -708,16 +782,22 @@ public class StructureDataTypeTest extends AbstractGenericTest { "pack(disabled)\n" + "Structure TestStruct {\n" + " 0 byte 1 field1 \"Comment1\"\n" + -// " 1 undefined 1 \"\"\n" + +// " 1 undefined 1 \"\"\n" + + " 2 char[0] 0 A \"\"\n" + + " 2 char[0] 0 B \"\"\n" + + " 2 char[0] 0 C \"\"\n" + + " 2 char[0] 0 D \"\"\n" + + " 2 char[0] 0 E \"\"\n" + " 2 int:3(0) 1 bf1 \"bf1Comment\"\n" + " 2 int:3(3) 1 bf2 \"bf2Comment\"\n" + " 2 int:15(6) 3 bf3 \"bf3Comment\"\n" + // " 5 undefined 1 \"\"\n" + - " 6 word 2 \"Comment2\"\n" + - " 8 dword 4 field3 \"\"\n" + - " 12 byte 1 field4 \"Comment4\"\n" + + " 6 char 1 XXX \"\"\n" + + " 7 word 2 \"Comment2\"\n" + + " 9 dword 4 field3 \"\"\n" + + " 13 byte 1 field4 \"Comment4\"\n" + "}\n" + - "Length: 13 Alignment: 1", struct); + "Length: 14 Alignment: 1", struct); //@formatter:on try { @@ -737,15 +817,47 @@ public class StructureDataTypeTest extends AbstractGenericTest { "Structure TestStruct {\n" + " 0 byte 1 field1 \"Comment1\"\n" + // " 1 undefined 1 \"\"\n" + + " 2 char[0] 0 A \"\"\n" + + " 2 char[0] 0 B \"\"\n" + + " 2 char[0] 0 C \"\"\n" + + " 2 char[0] 0 D \"\"\n" + + " 2 char[0] 0 E \"\"\n" + " 2 int:3(0) 1 bf1 \"bf1Comment\"\n" + " 2 int:3(3) 1 bf2 \"bf2Comment\"\n" + " 2 int:15(6) 3 bf3 \"bf3Comment\"\n" + " 4 int:11(5) 2 bf4 \"bf4Comment\"\n" + - " 6 word 2 \"Comment2\"\n" + - " 8 dword 4 field3 \"\"\n" + - " 12 byte 1 field4 \"Comment4\"\n" + + " 6 char 1 XXX \"\"\n" + + " 7 word 2 \"Comment2\"\n" + + " 9 dword 4 field3 \"\"\n" + + " 13 byte 1 field4 \"Comment4\"\n" + "}\n" + - "Length: 13 Alignment: 1", struct); + "Length: 14 Alignment: 1", struct); + //@formatter:on + + struct.insertBitFieldAt(2, 4, 0, IntegerDataType.dataType, 0, "z", "zero bitfield"); + + //@formatter:off + CompositeTestUtils.assertExpectedComposite(this, "/TestStruct\n" + + "pack(disabled)\n" + + "Structure TestStruct {\n" + + " 0 byte 1 field1 \"Comment1\"\n" + +// " 1 undefined 1 \"\"\n" + + " 2 char[0] 0 A \"\"\n" + + " 2 char[0] 0 B \"\"\n" + + " 2 char[0] 0 C \"\"\n" + + " 2 char[0] 0 D \"\"\n" + + " 2 char[0] 0 E \"\"\n" + + " 2 int:0(0) 0 \"zero bitfield\"\n" + // field name discarded + " 2 int:3(0) 1 bf1 \"bf1Comment\"\n" + + " 2 int:3(3) 1 bf2 \"bf2Comment\"\n" + + " 2 int:15(6) 3 bf3 \"bf3Comment\"\n" + + " 4 int:11(5) 2 bf4 \"bf4Comment\"\n" + + " 6 char 1 XXX \"\"\n" + + " 7 word 2 \"Comment2\"\n" + + " 9 dword 4 field3 \"\"\n" + + " 13 byte 1 field4 \"Comment4\"\n" + + "}\n" + + "Length: 14 Alignment: 1", struct); //@formatter:on } @@ -839,6 +951,26 @@ public class StructureDataTypeTest extends AbstractGenericTest { "}\n" + "Length: 13 Alignment: 1", struct); //@formatter:on + + struct.insertBitFieldAt(2, 4, 31, IntegerDataType.dataType, 0, "z", "zero bitfield"); + + //@formatter:off + CompositeTestUtils.assertExpectedComposite(this, "/TestStruct\n" + + "pack(disabled)\n" + + "Structure TestStruct {\n" + + " 0 byte 1 field1 \"Comment1\"\n" + +// " 1 undefined 1 \"\"\n" + + " 2 int:0(7) 0 \"zero bitfield\"\n" + // field name discarded + " 2 int:3(5) 1 bf1 \"bf1Comment\"\n" + + " 2 int:3(2) 1 bf2 \"bf2Comment\"\n" + + " 2 int:15(3) 3 bf3 \"bf3Comment\"\n" + + " 4 int:11(0) 2 bf4 \"bf4Comment\"\n" + + " 6 word 2 \"Comment2\"\n" + + " 8 dword 4 field3 \"\"\n" + + " 12 byte 1 field4 \"Comment4\"\n" + + "}\n" + + "Length: 13 Alignment: 1", struct); + //@formatter:on } @Test diff --git a/Ghidra/Features/Decompiler/src/decompile/cpp/constseq.cc b/Ghidra/Features/Decompiler/src/decompile/cpp/constseq.cc index d81f38d851..b5e31405eb 100644 --- a/Ghidra/Features/Decompiler/src/decompile/cpp/constseq.cc +++ b/Ghidra/Features/Decompiler/src/decompile/cpp/constseq.cc @@ -767,7 +767,7 @@ PcodeOp *HeapSequence::buildStringCopy(void) /// the initial input and final output are gathered. /// \param indirects will hold the INDIRECT ops attached to sequence STOREs /// \param pairs will hold Varnode pairs where the first in the pair is the input and the second is the output -void HeapSequence::gatherIndirectPairs(vector &indirects,vector &pairs) +void HeapSequence::gatherIndirectPairs(vector &indirects,vector &pairs) { for(int4 i=0;i &indirects,vectorisMark()) break; invn = defOp->getIn(0); } - pairs.push_back(invn); - pairs.push_back(outvn); - data.opUnsetOutput(op); + pairs.emplace_back(invn,outvn); } } for(int4 i=0;iclearMark(); } +bool HeapSequence::IndirectPair::compareOutput(const IndirectPair *a,const IndirectPair *b) + +{ + Varnode *vn1 = a->outVn; + Varnode *vn2 = b->outVn; + if (vn1->getSpace() != vn2->getSpace()) + return vn1->getSpace()->getIndex() < vn2->getSpace()->getIndex(); + if (vn1->getOffset() != vn2->getOffset()) + return vn1->getOffset() < vn2->getOffset(); + if (vn1->getSize() != vn2->getSize()) + return vn1->getSize() < vn2->getSize(); + return false; +} + +/// Its possible that INDIRECTs collected from different \e effect ops may share +/// the same output storage. Find any output Varnodes that share storage and +/// replace all their reads with a single representative Varnode. +/// \param pairs is the list of INDIRECT pairs +/// \return \b true if the deduplication succeeded +bool HeapSequence::deduplicatePairs(vector &pairs) + +{ + if (pairs.empty()) return true; + vector copy(pairs.size(),(IndirectPair *)0); + for(int4 i=0;ioutVn; + int4 overlap = head->outVn->characterizeOverlap(*vn); + if (overlap == 1) + return false; // Partial overlap + if (overlap == 2) { + if (copy[i]->inVn != head->inVn) { + return false; // Same storage coming from different sources + } + copy[i]->markDuplicate(); + dupCount += 1; // Found a duplicate, keep the same head for next iteration + } + else // No overlap, move to next headVn + head = copy[i]; + } + if (dupCount > 0) { + head = copy[0]; + for(int4 i=1;iisDuplicate()) { + data.totalReplace(copy[i]->outVn, head->outVn); + } + else + head = copy[i]; + } + } + return true; +} /// If the STORE pointer no longer has any other uses, remove the PTRADD producing it, recursively, /// up to the base pointer. INDIRECT ops surrounding any STORE that is removed are replaced with /// INDIRECTs around the user-op replacing the STOREs. +/// \param indirects are the list of INDIRECTs cause by the STOREs +/// \param indirectPairs are the flow pairs across the STOREs that need to be preserved /// \param replaceOp is the user-op replacement for the STOREs -void HeapSequence::removeStoreOps(PcodeOp *replaceOp) +void HeapSequence::removeStoreOps(vector &indirects,vector &indirectPairs,PcodeOp *replaceOp) { - vector indirects; - vector indirectPairs; vector scratch; - gatherIndirectPairs(indirects, indirectPairs); + for(int4 i=0;igetDef()); + } for(int4 i=0;igetAddr()); data.opSetOpcode(newInd, CPUI_INDIRECT); - data.opSetOutput(newInd,outvn); - data.opSetInput(newInd,invn,0); + data.opSetOutput(newInd,indirectPairs[i].outVn); + data.opSetInput(newInd,indirectPairs[i].inVn,0); data.opSetInput(newInd,data.newVarnodeIop(replaceOp),1); data.opInsertBefore(newInd, replaceOp); } @@ -871,10 +927,15 @@ HeapSequence::HeapSequence(Funcdata &fdata,Datatype *ct,PcodeOp *root) bool HeapSequence::transform(void) { + vector indirects; + vector indirectPairs; + gatherIndirectPairs(indirects, indirectPairs); + if (!deduplicatePairs(indirectPairs)) + return false; PcodeOp *memCpyOp = buildStringCopy(); if (memCpyOp == (PcodeOp *)0) return false; - removeStoreOps(memCpyOp); + removeStoreOps(indirects,indirectPairs,memCpyOp); return true; } diff --git a/Ghidra/Features/Decompiler/src/decompile/cpp/constseq.hh b/Ghidra/Features/Decompiler/src/decompile/cpp/constseq.hh index dbfc3c512a..5df6bf1b90 100644 --- a/Ghidra/Features/Decompiler/src/decompile/cpp/constseq.hh +++ b/Ghidra/Features/Decompiler/src/decompile/cpp/constseq.hh @@ -84,6 +84,16 @@ public: /// a single string into memory. If the transform() method is called, an explicit string is constructed, and /// the STOREs are replaced with a \b strncpy or similar CALLOTHER that takes the string as its source input. class HeapSequence : public ArraySequence { + /// \brief Helper class containing Varnode pairs that flow across a sequence of INDIRECTs + class IndirectPair { + public: + Varnode *inVn; ///< Input to INDIRECTs + Varnode *outVn; ///< Output of INDIRECTs + IndirectPair(Varnode *in,Varnode *out) { inVn = in; outVn = out; } ///< Constructor + void markDuplicate(void) { inVn = (Varnode *)0; } ///< Note that \b this is a duplicate of another pair + bool isDuplicate(void) const { return (inVn == (Varnode *)0); } ///< Return \b true if \b this is marked as a duplicate + static bool compareOutput(const IndirectPair *a,const IndirectPair *b); ///< Compare pairs by output storage + }; Varnode *basePointer; ///< Pointer that sequence is stored to uint8 baseOffset; ///< Offset relative to pointer to root STORE AddrSpace *storeSpace; ///< Address space being STOREed to @@ -98,8 +108,9 @@ class HeapSequence : public ArraySequence { bool testValue(PcodeOp *op); ///< Test if a STORE value has the matching form for the sequence bool collectStoreOps(void); ///< Collect ops STOREing into a memory region from the same root pointer PcodeOp *buildStringCopy(void); ///< Build the strncpy,wcsncpy, or memcpy function with string as input - void gatherIndirectPairs(vector &indirects,vector &pairs); - void removeStoreOps(PcodeOp *replaceOp); ///< Remove all STORE ops from the basic block + void gatherIndirectPairs(vector &indirects,vector &pairs); + bool deduplicatePairs(vector &pairs); ///< Find and eliminate duplicate INDIRECT pairs + void removeStoreOps(vector &indirects,vector &indirectPairs,PcodeOp *replaceOp); ///< Remove all STORE ops from the basic block public: HeapSequence(Funcdata &fdata,Datatype *ct,PcodeOp *root); bool transform(void); ///< Transform STOREs into a single memcpy user-op diff --git a/Ghidra/Features/Decompiler/src/decompile/cpp/typeop.cc b/Ghidra/Features/Decompiler/src/decompile/cpp/typeop.cc index 549b10e824..5197e3eefd 100644 --- a/Ghidra/Features/Decompiler/src/decompile/cpp/typeop.cc +++ b/Ghidra/Features/Decompiler/src/decompile/cpp/typeop.cc @@ -2254,7 +2254,13 @@ Datatype *TypeOpPtradd::getInputCast(const PcodeOp *op,int4 slot,const CastStrat // not the (possibly different) type of the HIGH Datatype *reqtype = op->getIn(0)->getTypeReadFacing(op); Datatype *curtype = op->getIn(0)->getHighTypeReadFacing(op); - return castStrategy->castStandard(reqtype,curtype,false,false); + if (reqtype->getMetatype() != TYPE_PTR) return reqtype; + if (curtype->getMetatype() != TYPE_PTR) return reqtype; + Datatype *reqbase = ((TypePointer *)reqtype)->getPtrTo(); // Go down exactly one level + Datatype *curbase = ((TypePointer *)curtype)->getPtrTo(); + if (reqbase->getAlignSize() == curbase->getAlignSize()) + return (Datatype *)0; + return reqtype; } return TypeOp::getInputCast(op,slot,castStrategy); } @@ -2318,7 +2324,24 @@ Datatype *TypeOpPtrsub::getInputCast(const PcodeOp *op,int4 slot,const CastStrat // not the (possibly different) type of the HIGH Datatype *reqtype = op->getIn(0)->getTypeReadFacing(op); Datatype *curtype = op->getIn(0)->getHighTypeReadFacing(op); - return castStrategy->castStandard(reqtype,curtype,false,false); + if (curtype == reqtype) + return (Datatype *)0; + if (reqtype->getMetatype() != TYPE_PTR) return reqtype; + if (curtype->getMetatype() != TYPE_PTR) return reqtype; + Datatype *reqbase = ((TypePointer *)reqtype)->getPtrTo(); // Go down exactly one level + Datatype *curbase = ((TypePointer *)curtype)->getPtrTo(); + if (curbase->getMetatype() == TYPE_ARRAY && reqbase->getMetatype() == TYPE_ARRAY) { + curbase = ((TypeArray *)curbase)->getBase(); + reqbase = ((TypeArray *)reqbase)->getBase(); + } + while(reqbase->getTypedef() != (Datatype *)0) + reqbase = reqbase->getTypedef(); + while(curbase->getTypedef() != (Datatype *)0) + curbase = curbase->getTypedef(); + + if (curbase == reqbase) + return (Datatype *)0; + return reqtype; } return TypeOp::getInputCast(op,slot,castStrategy); } diff --git a/Ghidra/Features/DecompilerDependent/src/main/java/ghidra/app/extension/datatype/finder/DtrfDbg.java b/Ghidra/Features/DecompilerDependent/src/main/java/ghidra/app/extension/datatype/finder/DtrfDbg.java index 5b1df370da..71cac2c807 100644 --- a/Ghidra/Features/DecompilerDependent/src/main/java/ghidra/app/extension/datatype/finder/DtrfDbg.java +++ b/Ghidra/Features/DecompilerDependent/src/main/java/ghidra/app/extension/datatype/finder/DtrfDbg.java @@ -31,6 +31,9 @@ import ghidra.util.Msg; * A package utility class to allow for tests to selectively enable debug output. This class is * used instead of generic logging with the intent that this class will be removed when the bug(s) * are fixed. + *

+ * Until {@link #enable()} is called, no data is recorded. Once enabled, all messages are buffered + * until a call to {@link #disable(boolean)} is made. */ class DtrfDbg { @@ -41,18 +44,24 @@ class DtrfDbg { private static Map> linesByFunction = new ConcurrentHashMap<>(); - DtrfDbg() { + private static volatile boolean isEnabled = false; + + private DtrfDbg() { // static class } static void enable() { debugBytes = new ByteArrayOutputStream(); debugWriter = new PrintWriter(debugBytes); + linesByFunction = new ConcurrentHashMap<>(); + isEnabled = true; } private static void close() { + isEnabled = false; debugWriter.close(); debugWriter = new NullPrintWriter(); + linesByFunction.clear(); } static void disable(boolean write) { @@ -91,11 +100,31 @@ class DtrfDbg { clientFilters.addAll(Arrays.asList(filters)); } + /** + * Stores a message to later be printed. + * + * @param f the function + * @param s the message + */ static void println(Function f, String s) { - linesByFunction.computeIfAbsent(f, ff -> new ArrayList<>()).add(s); + if (isEnabled) { + linesByFunction.computeIfAbsent(f, ff -> new ArrayList<>()).add(s); + } } + /** + * Stores a message to later be printed, filtering messages based on the 'client' parameter. + * + * @param f the function + * @param client the client + * @param s the message + * @see #setClientToStringFilters(String...) + */ static void println(Function f, Object client, String s) { + if (!isEnabled) { + return; + } + if (!passesFilter(client)) { return; } diff --git a/Ghidra/Features/GhidraServer/src/main/java/ghidra/server/remote/GhidraServer.java b/Ghidra/Features/GhidraServer/src/main/java/ghidra/server/remote/GhidraServer.java index 70839d598f..90421f4b4b 100644 --- a/Ghidra/Features/GhidraServer/src/main/java/ghidra/server/remote/GhidraServer.java +++ b/Ghidra/Features/GhidraServer/src/main/java/ghidra/server/remote/GhidraServer.java @@ -25,8 +25,7 @@ import java.rmi.registry.LocateRegistry; import java.rmi.registry.Registry; import java.rmi.server.*; import java.security.cert.CertificateException; -import java.util.Enumeration; -import java.util.List; +import java.util.*; import javax.rmi.ssl.SslRMIClientSocketFactory; import javax.rmi.ssl.SslRMIServerSocketFactory; @@ -752,11 +751,49 @@ public class GhidraServer extends UnicastRemoteObject implements GhidraServerHan // Ensure that remote access hostname is properly set for RMI registration String hostname = initRemoteAccessHostname(); - if (DefaultKeyManagerFactory.getPreferredKeyStore() == null) { + log.info("Ghidra Server " + Application.getApplicationVersion()); + log.info(" Server remote access address: " + hostname); + if (bindAddress == null) { + log.info(" Server listening on all interfaces"); + } + else { + log.info(" Server listening on interface: " + bindAddress.getHostAddress()); + } + + String preferredKeyStore = DefaultKeyManagerFactory.getPreferredKeyStore(); + if (preferredKeyStore == null) { + // keystore has not been identified - use self-signed certificate + log.info(" Generating self-signed certificate..."); + log.info(" Subject Alternative Names:"); + log.info(" " + hostname); + DefaultKeyManagerFactory.setDefaultIdentity(new X500Principal("CN=GhidraServer")); DefaultKeyManagerFactory.addSubjectAlternativeName(hostname); + + // Collect alternate hostnames for inclusion in certificate + Set altNames = new TreeSet<>(); + Enumeration nets = NetworkInterface.getNetworkInterfaces(); + while (nets.hasMoreElements()) { + NetworkInterface netint = nets.nextElement(); + Enumeration addrs = netint.getInetAddresses(); + while (addrs.hasMoreElements()) { + InetAddress addr = addrs.nextElement(); + altNames.add(addr.getHostAddress()); + altNames.add(addr.getHostName()); + altNames.add(addr.getCanonicalHostName()); + } + } + altNames.remove(hostname); + for (String name : altNames) { + log.info(" " + name); + DefaultKeyManagerFactory.addSubjectAlternativeName(name); + } } + else { + log.info(" Using server certificate keystore: " + preferredKeyStore); + } + if (!DefaultKeyManagerFactory.initialize()) { log.fatal("Failed to initialize PKI/SSL keystore"); System.exit(0); @@ -769,14 +806,7 @@ public class GhidraServer extends UnicastRemoteObject implements GhidraServerHan // localhost.getCanonicalHostName() + ":" + classSvrPort + "/"; // System.setProperty(RMI_CODEBASE_PROPERTY, codeBaseProp); - log.info("Ghidra Server " + Application.getApplicationVersion()); - log.info(" Server remote access address: " + hostname); - if (bindAddress == null) { - log.info(" Server listening on all interfaces"); - } - else { - log.info(" Server listening on interface: " + bindAddress.getHostAddress()); - } + log.info(" RMI Registry port: " + ServerPortFactory.getRMIRegistryPort()); log.info(" RMI SSL port: " + ServerPortFactory.getRMISSLPort()); log.info(" Block Stream port: " + ServerPortFactory.getStreamPort()); diff --git a/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/plugin/core/analysis/GnuDemanglerAnalyzer.java b/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/plugin/core/analysis/GnuDemanglerAnalyzer.java index 31beb384f0..a35d6f9431 100644 --- a/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/plugin/core/analysis/GnuDemanglerAnalyzer.java +++ b/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/plugin/core/analysis/GnuDemanglerAnalyzer.java @@ -66,10 +66,15 @@ public class GnuDemanglerAnalyzer extends AbstractDemanglerAnalyzer { private static final String OPTION_DESCRIPTION_DEMANGLER_FORMAT = "The demangling format to use"; + static final String OPTION_NAME_TIMEOUT_SECONDS = "Timeout (seconds)"; + private static final String OPTION_DESCRIPTION_TIMEOUT_SECONDS = + "The maximum amount of seconds to spend demangling a string"; + private boolean applyFunctionSignature = true; private boolean applyCallingConvention = true; private boolean demangleOnlyKnownPatterns = false; private boolean useStandardReplacements = true; + private long timeoutSeconds = GnuDemanglerOptions.DEFAULT_TIMEOUT_SECONDS; private GnuDemanglerFormat demanglerFormat = GnuDemanglerFormat.AUTO; private boolean useDeprecatedDemangler = false; @@ -110,6 +115,8 @@ public class GnuDemanglerAnalyzer extends AbstractDemanglerAnalyzer { demanglerFormat, help, OPTION_DESCRIPTION_DEMANGLER_FORMAT, () -> optionsEditor.getFormatEditor()); + options.registerOption(OPTION_NAME_TIMEOUT_SECONDS, timeoutSeconds, help, + OPTION_DESCRIPTION_TIMEOUT_SECONDS); } @Override @@ -123,6 +130,8 @@ public class GnuDemanglerAnalyzer extends AbstractDemanglerAnalyzer { useStandardReplacements = options.getBoolean(OPTION_NAME_DEMANGLE_USE_STANDARD_REPLACEMENTS, useStandardReplacements); + timeoutSeconds = options.getLong(OPTION_NAME_TIMEOUT_SECONDS, + GnuDemanglerOptions.DEFAULT_TIMEOUT_SECONDS); demanglerFormat = options.getEnum(OPTION_NAME_DEMANGLER_FORMAT, GnuDemanglerFormat.AUTO); useDeprecatedDemangler = options.getBoolean(OPTION_NAME_USE_DEPRECATED_DEMANGLER, useDeprecatedDemangler); @@ -131,7 +140,7 @@ public class GnuDemanglerAnalyzer extends AbstractDemanglerAnalyzer { @Override protected DemanglerOptions getOptions() { GnuDemanglerOptions options = - new GnuDemanglerOptions(demanglerFormat, useDeprecatedDemangler); + new GnuDemanglerOptions(demanglerFormat, useDeprecatedDemangler, timeoutSeconds); options.setDoDisassembly(true); options.setApplySignature(applyFunctionSignature); options.setApplyCallingConvention(applyCallingConvention); diff --git a/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/util/demangler/gnu/GnuDemangler.java b/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/util/demangler/gnu/GnuDemangler.java index 4c051e89f2..00de9ff4f3 100644 --- a/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/util/demangler/gnu/GnuDemangler.java +++ b/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/util/demangler/gnu/GnuDemangler.java @@ -108,7 +108,7 @@ public class GnuDemangler implements Demangler { try { GnuDemanglerNativeProcess process = getNativeProcess(options); - String demangled = process.demangle(mangled); + String demangled = process.demangle(mangled, options.getTimeoutSeconds()); if (demangled == null) { throw new DemangledException(false); } diff --git a/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/util/demangler/gnu/GnuDemanglerNativeProcess.java b/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/util/demangler/gnu/GnuDemanglerNativeProcess.java index 8453d87ffd..8e80ff35e5 100644 --- a/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/util/demangler/gnu/GnuDemanglerNativeProcess.java +++ b/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/util/demangler/gnu/GnuDemanglerNativeProcess.java @@ -15,16 +15,34 @@ */ package ghidra.app.util.demangler.gnu; -import java.io.*; +import java.io.BufferedReader; +import java.io.File; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.io.OutputStream; +import java.io.PrintWriter; +import java.io.UncheckedIOException; import java.nio.charset.Charset; -import java.util.*; +import java.util.Arrays; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.TimeoutException; import org.apache.commons.io.FilenameUtils; import org.apache.commons.io.IOUtils; import org.apache.commons.lang3.ArrayUtils; import org.apache.commons.lang3.StringUtils; -import ghidra.framework.*; +import com.google.common.util.concurrent.SimpleTimeLimiter; + +import ghidra.app.plugin.core.analysis.AutoAnalysisManager; +import ghidra.framework.Application; +import ghidra.framework.OSFileNotFoundException; +import ghidra.framework.Platform; import ghidra.util.Msg; /** @@ -113,16 +131,39 @@ public class GnuDemanglerNativeProcess { createProcess(); } + /** + * {@return the demangled string} + * + * @param mangled The string to demangle + * @throws IOException if an IO-related error occurred + */ public synchronized String demangle(String mangled) throws IOException { + return demangle(mangled, true, null); + } + + /** + * {@return the demangled string} + * + * @param mangled The string to demangle + * @param timeoutSeconds The number of seconds to attempt the demangle, or {@code null} for no + * timeout + * @throws IOException if a timeout or IO-related error occurred + */ + public synchronized String demangle(String mangled, Long timeoutSeconds) throws IOException { if (isDisposed) { throw new IOException("Demangled process has been terminated."); } - return demangle(mangled, true); + return demangle(mangled, true, timeoutSeconds); } - private String demangle(String mangled, boolean restart) throws IOException { + private String demangle(String mangled, boolean restart, Long timeoutSeconds) + throws IOException { try { - return doDemangle(mangled); + return doDemangle(mangled, timeoutSeconds); + } + catch (TimeoutException e) { + dispose(); + throw new IOException("Timeout reached", e); } catch (IOException e) { dispose(); @@ -130,14 +171,25 @@ public class GnuDemanglerNativeProcess { throw new IOException("Demangler process is not running.", e); } createProcess(); - return demangle(mangled, false); + return demangle(mangled, false, timeoutSeconds); } } - private String doDemangle(String mangled) throws IOException { + private String doDemangle(String mangled, Long timeoutSeconds) + throws TimeoutException, IOException { writer.println(mangled); writer.flush(); - return reader.readLine(); + try { + return timeoutSeconds != null + ? SimpleTimeLimiter + .create(AutoAnalysisManager.getSharedAnalsysThreadPool() + .getExecutorService()) + .callWithTimeout(reader::readLine, timeoutSeconds, TimeUnit.SECONDS) + : reader.readLine(); + } + catch (ExecutionException | InterruptedException e) { + throw new IOException(e); + } } public void dispose() { @@ -161,6 +213,7 @@ public class GnuDemanglerNativeProcess { } } + @SuppressWarnings("resource") private void createProcess() throws IOException { String[] command = buildCommand(); @@ -227,9 +280,14 @@ public class GnuDemanglerNativeProcess { // Send a test string over and read the result. If the test string is blank, then // there was an error. // - String testResult = doDemangle("test"); - if (!StringUtils.isBlank(testResult)) { - return; + try { + String testResult = doDemangle("test", null); + if (!StringUtils.isBlank(testResult)) { + return; + } + } + catch (TimeoutException e) { + throw new IOException(e); } InputStream err = process.getErrorStream(); diff --git a/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/util/demangler/gnu/GnuDemanglerOptions.java b/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/util/demangler/gnu/GnuDemanglerOptions.java index 26f79880bc..23fcbe4884 100644 --- a/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/util/demangler/gnu/GnuDemanglerOptions.java +++ b/Ghidra/Features/GnuDemangler/src/main/java/ghidra/app/util/demangler/gnu/GnuDemanglerOptions.java @@ -44,9 +44,15 @@ public class GnuDemanglerOptions extends DemanglerOptions { */ public static final String GNU_DEMANGLER_DEFAULT = GNU_DEMANGLER_V2_41; + /** + * The default GNU demangler timeout (in seconds) + */ + public static final long DEFAULT_TIMEOUT_SECONDS = 3; + private final GnuDemanglerFormat format; private final boolean isDeprecated; private boolean useStandardReplacements; + private long timeout; /** * Default constructor to use the modern demangler with auto-detect for the format. This @@ -75,9 +81,25 @@ public class GnuDemanglerOptions extends DemanglerOptions { * demangler */ public GnuDemanglerOptions(GnuDemanglerFormat format, boolean isDeprecated) { + this(format, isDeprecated, GnuDemanglerOptions.DEFAULT_TIMEOUT_SECONDS); + } + + /** + * Constructor to specify the format to use, whether to prefer the deprecated format when + * both deprecated and modern are available, and the timeout + * + * @param format the format + * @param isDeprecated true if the format is not available in the modern demangler + * @param timeoutSeconds the demangler timeout in seconds + * @throws IllegalArgumentException if the given format is not available in the deprecated + * demangler + */ + public GnuDemanglerOptions(GnuDemanglerFormat format, boolean isDeprecated, + long timeoutSeconds) { this.format = format; this.isDeprecated = isDeprecated; this.useStandardReplacements = true; + this.timeout = timeoutSeconds; if (!format.isAvailable(isDeprecated)) { throw new IllegalArgumentException( format.name() + " is not available in the " + getDemanglerName()); @@ -91,30 +113,32 @@ public class GnuDemanglerOptions extends DemanglerOptions { public GnuDemanglerOptions(DemanglerOptions copy) { super(copy); - if (copy instanceof GnuDemanglerOptions) { - GnuDemanglerOptions gCopy = (GnuDemanglerOptions) copy; + if (copy instanceof GnuDemanglerOptions gCopy) { format = gCopy.format; isDeprecated = gCopy.isDeprecated; + timeout = gCopy.timeout; } else { format = GnuDemanglerFormat.AUTO; isDeprecated = false; + timeout = DEFAULT_TIMEOUT_SECONDS; } this.useStandardReplacements = true; } private GnuDemanglerOptions(GnuDemanglerOptions copy, GnuDemanglerFormat format, - boolean deprecated) { - this(copy, format, deprecated, true); + boolean deprecated, long timeoutSeconds) { + this(copy, format, deprecated, true, timeoutSeconds); } private GnuDemanglerOptions(GnuDemanglerOptions copy, GnuDemanglerFormat format, - boolean deprecated, boolean useStandardReplacements) { + boolean deprecated, boolean useStandardReplacements, long timeoutSeconds) { super(copy); this.format = format; this.isDeprecated = deprecated; this.useStandardReplacements = useStandardReplacements; + this.timeout = timeoutSeconds; } /** @@ -150,7 +174,7 @@ public class GnuDemanglerOptions extends DemanglerOptions { return this; } if (demanglerFormat.isAvailable(useDeprecated)) { - return new GnuDemanglerOptions(this, demanglerFormat, useDeprecated); + return new GnuDemanglerOptions(this, demanglerFormat, useDeprecated, this.timeout); } throw new IllegalArgumentException( demanglerFormat.name() + " is not available in the " + getDemanglerName()); @@ -186,6 +210,13 @@ public class GnuDemanglerOptions extends DemanglerOptions { return useStandardReplacements; } + /** + * {@return the demangler timeout (in seconds)} + */ + public long getTimeoutSeconds() { + return timeout; + } + @Override public String toString() { //@formatter:off @@ -194,6 +225,7 @@ public class GnuDemanglerOptions extends DemanglerOptions { "\tapplySignature: " + applySignature() + ",\n" + "\tuseStandardReplacements: " + useStandardReplacements + ",\n" + "\tdemangleOnlyKnownPatterns: " + demangleOnlyKnownPatterns() + ",\n" + + "\ttimeout (sec): " + timeout + ",\n" + "\tdemanglerName: " + getDemanglerName() + ",\n" + "\tdemanglerApplicationArguments: " + getDemanglerApplicationArguments() + ",\n" + "}"; diff --git a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/data/CompositeDB.java b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/data/CompositeDB.java index 66b2002171..74237059a0 100644 --- a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/data/CompositeDB.java +++ b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/data/CompositeDB.java @@ -30,7 +30,7 @@ import ghidra.util.UniversalID; import ghidra.util.exception.AssertException; /** - * Database implementation for a structure or union. + * {@link CompositeDB} provides an abstract database implementation for a structure or union. */ abstract class CompositeDB extends DataTypeDB implements CompositeInternal { diff --git a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/data/StructureDB.java b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/data/StructureDB.java index 32b2579f1d..5c14268e8e 100644 --- a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/data/StructureDB.java +++ b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/data/StructureDB.java @@ -1007,6 +1007,26 @@ class StructureDB extends CompositeDB implements StructureInternal { return index; } + /** + * Find insertion index such that the index falls after all zero-length components at the + * specified offset and before any bitfields at that offset. + * + * @param index any defined component index which contains offset. + * @param offset offset within structure. + * @return index of first non-zero-length defined checking in the forward direction. + */ + private int afterNonZeroComponentsAtOffset(int index, int offset) { + int maxIndex = components.size(); + while (index < maxIndex) { + DataTypeComponentDB dtc = components.get(index); + if (dtc.getOffset() != offset || dtc.getLength() != 0) { + break; + } + ++index; + } + return index; + } + /** * Identify defined-component index of the first non-zero-length component which contains the * specified offset. If only zero-length components exist, the last zero-length component which @@ -1322,14 +1342,20 @@ class StructureDB extends CompositeDB implements StructureInternal { structLength = offset; } + // Any component insert at an offset should be placed after any zero-length components + // at the same offset but before any non-zero-length component. + int index = Collections.binarySearch(components, Integer.valueOf(offset), OffsetComparator.INSTANCE); int additionalShift = 0; if (index >= 0) { index = backupToFirstComponentContainingOffset(index, offset); - DataTypeComponentDB dtc = components.get(index); - additionalShift = offset - dtc.getOffset(); + index = afterNonZeroComponentsAtOffset(index, offset); + if (index < components.size()) { + DataTypeComponentDB dtc = components.get(index); + additionalShift = offset - dtc.getOffset(); + } } else { index = -index - 1; diff --git a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/BitFieldDataType.java b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/BitFieldDataType.java index bcccdca19e..0c3c439890 100644 --- a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/BitFieldDataType.java +++ b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/BitFieldDataType.java @@ -184,6 +184,11 @@ public class BitFieldDataType extends AbstractDataType { * Get the packing storage size in bytes associated with this bit-field which may be * larger than the base type associated with the fields original definition. * Returned value is the same as {@link #getLength()}. + *

+ * NOTE: Bitfields with a bit-size of zero will report a storage size of 1, although + * {@link #isZeroLength()} will return true. This is consistent with other datatypes which + * support a zero-length {@link DataTypeComponent} such as a zero-element Array. + * * @return packing storage size in bytes */ public int getStorageSize() { @@ -331,6 +336,9 @@ public class BitFieldDataType extends AbstractDataType { } } + /** + * @see #getStorageSize() + */ @Override public int getLength() { return storageSize; diff --git a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/DataTypeComponent.java b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/DataTypeComponent.java index dd1a63388e..33f83fc764 100644 --- a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/DataTypeComponent.java +++ b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/DataTypeComponent.java @@ -192,7 +192,7 @@ public interface DataTypeComponent { if (dataType instanceof Array) { return true; } - // assumes undefined types will ultimately have a non-zero length + // assumes not-yet-defined types will ultimately have a non-zero length return !dataType.isNotYetDefined(); } return false; diff --git a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/DataTypeComponentImpl.java b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/DataTypeComponentImpl.java index 4519b49a1b..0226062d05 100644 --- a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/DataTypeComponentImpl.java +++ b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/DataTypeComponentImpl.java @@ -61,6 +61,9 @@ public class DataTypeComponentImpl implements InternalDataTypeComponent, Seriali this.fieldName = InternalDataTypeComponent.cleanupFieldName(fieldName); setDataType(dataType); setComment(comment); + if (isZeroBitFieldComponent()) { + this.length = 0; // previously stored as 1, force to 0 + } } /** diff --git a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/Structure.java b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/Structure.java index 4a28743832..132f68a040 100644 --- a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/Structure.java +++ b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/Structure.java @@ -240,9 +240,16 @@ public interface Structure extends Composite { /** * Inserts a new datatype at the specified offset into this structure. Inserting a component * will cause any conflicting components to shift down to the extent necessary to avoid a - * conflict. + * conflict. The overall structure length will always increase when a non-zero-length + * component is inserted. NOTE: bitfields may share an offset with other bitfields and + * zero-length components. *

- * This method does not support bit-field insertions which must use the method + * Any component insert at an offset will be placed after any zero-length components + * at the same offset but before any non-zero-length components. The components which + * fall after the insertion point will have there ordinal incremented and offset + * adjusted as needed. + *

+ * This method will defer bit-field insertions to the method * {@link #insertBitFieldAt(int, int, int, DataType, int, String, String)}. * * @param offset the byte offset into the structure where the new datatype is to be inserted. @@ -389,7 +396,8 @@ public interface Structure extends Composite { /** * Replaces all components containing the specified byte offset with a new component using the * specified datatype, length, name and comment. If the offset corresponds to a bit-field - * more than one component may be consumed by this replacement. + * more than one component may be consumed by this replacement. In general, this method + * should not be used to replace bitfield components. *

* This method may not be used to replace a zero-length component since there may be any number * of zero-length components at the same offset. If the only defined component(s) at the specified diff --git a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/StructureDataType.java b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/StructureDataType.java index 5ff037ffba..171fc77449 100644 --- a/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/StructureDataType.java +++ b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/data/StructureDataType.java @@ -523,14 +523,20 @@ public class StructureDataType extends CompositeDataTypeImpl implements Structur structLength = offset; } + // Any component insert at an offset should be placed after any zero-length components + // at the same offset but before any non-zero-length component. + int index = Collections.binarySearch(components, Integer.valueOf(offset), OffsetComparator.INSTANCE); int additionalShift = 0; if (index >= 0) { index = backupToFirstComponentContainingOffset(index, offset); - DataTypeComponent dtc = components.get(index); - additionalShift = offset - dtc.getOffset(); + index = afterNonZeroComponentsAtOffset(index, offset); + if (index < components.size()) { + DataTypeComponentImpl dtc = components.get(index); + additionalShift = offset - dtc.getOffset(); + } } else { index = -index - 1; @@ -898,6 +904,26 @@ public class StructureDataType extends CompositeDataTypeImpl implements Structur return index; } + /** + * Find insertion index such that the index falls after all zero-length components at the + * specified offset and before any bitfields at that offset. + * + * @param index any defined component index which contains offset. + * @param offset offset within structure. + * @return index of first non-zero-length defined checking in the forward direction. + */ + private int afterNonZeroComponentsAtOffset(int index, int offset) { + int maxIndex = components.size(); + while (index < maxIndex) { + DataTypeComponentImpl dtc = components.get(index); + if (dtc.getOffset() != offset || dtc.getLength() != 0) { + break; + } + ++index; + } + return index; + } + /** * Identify defined-component index of the first non-zero-length component which contains the specified offset. * If only zero-length components exist, the last zero-length component which contains the offset will be returned. diff --git a/Ghidra/Framework/SoftwareModeling/src/test/java/ghidra/program/database/data/StructureDBTest.java b/Ghidra/Framework/SoftwareModeling/src/test/java/ghidra/program/database/data/StructureDBTest.java index 4a5e80f535..03adfc1d70 100644 --- a/Ghidra/Framework/SoftwareModeling/src/test/java/ghidra/program/database/data/StructureDBTest.java +++ b/Ghidra/Framework/SoftwareModeling/src/test/java/ghidra/program/database/data/StructureDBTest.java @@ -364,6 +364,74 @@ public class StructureDBTest extends AbstractGenericTest { assertEquals(ByteDataType.class, comps[2].getDataType().getClass()); } + @Test + public void testInsertAtSameOffset1() { + + DataType zeroDt = new ArrayDataType(ByteDataType.dataType, 0); + + struct = createStructure("Test", 100); + assertFalse(struct.isPackingEnabled()); + + struct.insertAtOffset(0, zeroDt, -1, "a", "comment a"); + struct.insertAtOffset(0, zeroDt, -1, "b", "comment b"); + struct.insertAtOffset(0, WordDataType.dataType, -1, "c", "comment c"); + + DataTypeComponentDB[] definedComponents = struct.getDefinedComponents(); + assertEquals("a", definedComponents[0].getFieldName()); + assertEquals("b", definedComponents[1].getFieldName()); + assertEquals("c", definedComponents[2].getFieldName()); + } + + @Test + public void testInsertAtSameOffset2() { + + DataType zeroDt = new ArrayDataType(ByteDataType.dataType, 0); + + struct = createStructure("Test", 100); + assertFalse(struct.isPackingEnabled()); + + struct.insertAtOffset(0, WordDataType.dataType, -1, "c", "comment c"); + struct.insertAtOffset(0, zeroDt, -1, "a", "comment a"); + struct.insertAtOffset(0, zeroDt, -1, "b", "comment b"); + + DataTypeComponentDB[] definedComponents = struct.getDefinedComponents(); + assertEquals("a", definedComponents[0].getFieldName()); + assertEquals("b", definedComponents[1].getFieldName()); + assertEquals("c", definedComponents[2].getFieldName()); + } + + @Test + public void testInsertAtEndOffset() { + + DataType zeroDt = new ArrayDataType(ByteDataType.dataType, 0); + + struct = createStructure("Test2", 100); + assertFalse(struct.isPackingEnabled()); + + struct.insertAtOffset(100, zeroDt, -1, "a", "comment a"); + struct.insertAtOffset(100, zeroDt, -1, "b", "comment b"); + + assertEquals("/Test2\n" + + "pack(disabled)\n" + + "Structure Test2 {\n" + + " 100 byte[0] 0 a \"comment a\"\n" + + " 100 byte[0] 0 b \"comment b\"\n" + + "}\n" + + "Length: 100 Alignment: 1\n", struct.toString()); + + // Insert non-zero-length component will increase struct size + struct.insertAtOffset(100, WordDataType.dataType, -1, "c", "comment c"); + + assertEquals("/Test2\n" + + "pack(disabled)\n" + + "Structure Test2 {\n" + + " 100 byte[0] 0 a \"comment a\"\n" + + " 100 byte[0] 0 b \"comment b\"\n" + + " 100 word 2 c \"comment c\"\n" + + "}\n" + + "Length: 102 Alignment: 1\n", struct.toString()); + } + // test inserting at offset 0 @Test public void testInsertAtOffset() { @@ -485,32 +553,18 @@ public class StructureDBTest extends AbstractGenericTest { Array zeroArray = new ArrayDataType(FloatDataType.dataType, 0, -1); struct.insertAtOffset(2, zeroArray, -1); struct.insertAtOffset(2, FloatDataType.dataType, -1); - assertEquals(13, struct.getLength()); - - DataTypeComponent[] comps = struct.getDefinedComponents(); - - assertEquals(6, comps.length); - - assertEquals(0, comps[0].getOffset()); - assertEquals(0, comps[0].getOrdinal()); - assertEquals(ByteDataType.class, comps[0].getDataType().getClass()); - - assertEquals(2, comps[1].getOffset()); - assertEquals(2, comps[1].getOrdinal()); - assertEquals(FloatDataType.class, comps[1].getDataType().getClass()); - - assertEquals(6, comps[2].getOffset()); - assertEquals(3, comps[2].getOrdinal()); - assertTrue(zeroArray.isEquivalent(comps[2].getDataType())); - - assertEquals(6, comps[3].getOffset()); - assertEquals(4, comps[3].getOrdinal()); - assertEquals(WordDataType.class, comps[3].getDataType().getClass()); - - assertEquals(8, comps[4].getOffset()); - assertEquals(5, comps[4].getOrdinal()); - assertEquals(DWordDataType.class, comps[4].getDataType().getClass()); + assertEquals("/Test\n" + + "pack(disabled)\n" + + "Structure Test {\n" + + " 0 byte 1 field1 \"Comment1\"\n" + + " 2 float[0] 0 \"\"\n" + + " 2 float 4 \"\"\n" + + " 6 word 2 \"Comment2\"\n" + + " 8 dword 4 field3 \"\"\n" + + " 12 byte 1 field4 \"Comment4\"\n" + + "}\n" + + "Length: 13 Alignment: 1\n", struct.toString()); } @Test @@ -815,6 +869,154 @@ public class StructureDBTest extends AbstractGenericTest { //@formatter:on } + @Test + public void testInsertBitFieldAtLittleEndian2() throws Exception { + + Array zeroArray = new ArrayDataType(CharDataType.dataType, 0, -1); + struct.insertAtOffset(2, zeroArray, -1, "A", null); + struct.insertAtOffset(2, zeroArray, -1, "B", null); + struct.insertAtOffset(2, zeroArray, -1, "C", null); + struct.insertAtOffset(2, zeroArray, -1, "D", null); + struct.insertAtOffset(2, zeroArray, -1, "E", null); + struct.insertAtOffset(2, CharDataType.dataType, -1, "XXX", null); + + struct.insertBitFieldAt(2, 4, 0, IntegerDataType.dataType, 3, "bf1", "bf1Comment"); + + //@formatter:off + CompositeTestUtils.assertExpectedComposite(this, "/Test\n" + + "pack(disabled)\n" + + "Structure Test {\n" + + " 0 byte 1 field1 \"Comment1\"\n" + +// " 1 undefined 1 \"\"\n" + + " 2 char[0] 0 A \"\"\n" + + " 2 char[0] 0 B \"\"\n" + + " 2 char[0] 0 C \"\"\n" + + " 2 char[0] 0 D \"\"\n" + + " 2 char[0] 0 E \"\"\n" + + " 2 int:3(0) 1 bf1 \"bf1Comment\"\n" + +// " 3 undefined 1 \"\"\n" + +// " 4 undefined 1 \"\"\n" + +// " 5 undefined 1 \"\"\n" + + " 6 char 1 XXX \"\"\n" + + " 7 word 2 \"Comment2\"\n" + + " 9 dword 4 field3 \"\"\n" + + " 13 byte 1 field4 \"Comment4\"\n" + + "}\n" + + "Length: 14 Alignment: 1", struct); + //@formatter:on + + struct.insertBitFieldAt(2, 4, 3, IntegerDataType.dataType, 3, "bf2", "bf2Comment"); + + //@formatter:off + CompositeTestUtils.assertExpectedComposite(this, "/Test\n" + + "pack(disabled)\n" + + "Structure Test {\n" + + " 0 byte 1 field1 \"Comment1\"\n" + +// " 1 undefined 1 \"\"\n" + + " 2 char[0] 0 A \"\"\n" + + " 2 char[0] 0 B \"\"\n" + + " 2 char[0] 0 C \"\"\n" + + " 2 char[0] 0 D \"\"\n" + + " 2 char[0] 0 E \"\"\n" + + " 2 int:3(0) 1 bf1 \"bf1Comment\"\n" + + " 2 int:3(3) 1 bf2 \"bf2Comment\"\n" + +// " 3 undefined 1 \"\"\n" + +// " 4 undefined 1 \"\"\n" + +// " 5 undefined 1 \"\"\n" + + " 6 char 1 XXX \"\"\n" + + " 7 word 2 \"Comment2\"\n" + + " 9 dword 4 field3 \"\"\n" + + " 13 byte 1 field4 \"Comment4\"\n" + + "}\n" + + "Length: 14 Alignment: 1", struct); + //@formatter:on + + struct.insertBitFieldAt(2, 4, 6, IntegerDataType.dataType, 15, "bf3", "bf3Comment"); + + //@formatter:off + CompositeTestUtils.assertExpectedComposite(this, "/Test\n" + + "pack(disabled)\n" + + "Structure Test {\n" + + " 0 byte 1 field1 \"Comment1\"\n" + +// " 1 undefined 1 \"\"\n" + + " 2 char[0] 0 A \"\"\n" + + " 2 char[0] 0 B \"\"\n" + + " 2 char[0] 0 C \"\"\n" + + " 2 char[0] 0 D \"\"\n" + + " 2 char[0] 0 E \"\"\n" + + " 2 int:3(0) 1 bf1 \"bf1Comment\"\n" + + " 2 int:3(3) 1 bf2 \"bf2Comment\"\n" + + " 2 int:15(6) 3 bf3 \"bf3Comment\"\n" + +// " 5 undefined 1 \"\"\n" + + " 6 char 1 XXX \"\"\n" + + " 7 word 2 \"Comment2\"\n" + + " 9 dword 4 field3 \"\"\n" + + " 13 byte 1 field4 \"Comment4\"\n" + + "}\n" + + "Length: 14 Alignment: 1", struct); + //@formatter:on + + try { + struct.insertBitFieldAt(2, 4, 21, IntegerDataType.dataType, 12, "bf4", "bf4Comment"); + fail( + "expected - IllegalArgumentException: Bitfield does not fit within specified constraints"); + } + catch (IllegalArgumentException e) { + // expected + } + + struct.insertBitFieldAt(2, 4, 21, IntegerDataType.dataType, 11, "bf4", "bf4Comment"); + + //@formatter:off + CompositeTestUtils.assertExpectedComposite(this, "/Test\n" + + "pack(disabled)\n" + + "Structure Test {\n" + + " 0 byte 1 field1 \"Comment1\"\n" + +// " 1 undefined 1 \"\"\n" + + " 2 char[0] 0 A \"\"\n" + + " 2 char[0] 0 B \"\"\n" + + " 2 char[0] 0 C \"\"\n" + + " 2 char[0] 0 D \"\"\n" + + " 2 char[0] 0 E \"\"\n" + + " 2 int:3(0) 1 bf1 \"bf1Comment\"\n" + + " 2 int:3(3) 1 bf2 \"bf2Comment\"\n" + + " 2 int:15(6) 3 bf3 \"bf3Comment\"\n" + + " 4 int:11(5) 2 bf4 \"bf4Comment\"\n" + + " 6 char 1 XXX \"\"\n" + + " 7 word 2 \"Comment2\"\n" + + " 9 dword 4 field3 \"\"\n" + + " 13 byte 1 field4 \"Comment4\"\n" + + "}\n" + + "Length: 14 Alignment: 1", struct); + //@formatter:on + + struct.insertBitFieldAt(2, 4, 0, IntegerDataType.dataType, 0, "z", "zero bitfield"); + + //@formatter:off + CompositeTestUtils.assertExpectedComposite(this, "/Test\n" + + "pack(disabled)\n" + + "Structure Test {\n" + + " 0 byte 1 field1 \"Comment1\"\n" + +// " 1 undefined 1 \"\"\n" + + " 2 char[0] 0 A \"\"\n" + + " 2 char[0] 0 B \"\"\n" + + " 2 char[0] 0 C \"\"\n" + + " 2 char[0] 0 D \"\"\n" + + " 2 char[0] 0 E \"\"\n" + + " 2 int:0(0) 0 \"zero bitfield\"\n" + // field name discarded + " 2 int:3(0) 1 bf1 \"bf1Comment\"\n" + + " 2 int:3(3) 1 bf2 \"bf2Comment\"\n" + + " 2 int:15(6) 3 bf3 \"bf3Comment\"\n" + + " 4 int:11(5) 2 bf4 \"bf4Comment\"\n" + + " 6 char 1 XXX \"\"\n" + + " 7 word 2 \"Comment2\"\n" + + " 9 dword 4 field3 \"\"\n" + + " 13 byte 1 field4 \"Comment4\"\n" + + "}\n" + + "Length: 14 Alignment: 1", struct); + //@formatter:on + } + @Test public void testInsertBitFieldAtBigEndian() throws Exception { diff --git a/Ghidra/Framework/Utility/src/main/java/generic/concurrent/GThreadPool.java b/Ghidra/Framework/Utility/src/main/java/generic/concurrent/GThreadPool.java index eb8957a35b..a122f4cd51 100644 --- a/Ghidra/Framework/Utility/src/main/java/generic/concurrent/GThreadPool.java +++ b/Ghidra/Framework/Utility/src/main/java/generic/concurrent/GThreadPool.java @@ -188,6 +188,21 @@ public class GThreadPool { public Executor getExecutor() { return executor; } + + + /** + * Returns the {@link ExecutorService} used by this thread pool. + * + *

Note: normal usage of this thread pool contraindicates accessing the executor service of + * this pool. For managing your own jobs, you should use the method on this class directly. + * The intent of this method is to provide access to the executor service so that it may be + * passed to other asynchronous APIs. + * + * @return the executor service + */ + public ExecutorService getExecutorService() { + return executor; + } //================================================================================================== // Inner Classes diff --git a/Ghidra/Processors/8051/data/languages/80251.sinc b/Ghidra/Processors/8051/data/languages/80251.sinc index 2658335ab4..97662df206 100644 --- a/Ghidra/Processors/8051/data/languages/80251.sinc +++ b/Ghidra/Processors/8051/data/languages/80251.sinc @@ -15,6 +15,7 @@ define token srcDestByte (8) drk03 = (0,3) # constraint bits d7 = (7,7) + d67 = (6,7) d57 = (5,7) d47 = (4,7) s3 = (3,3) @@ -36,6 +37,7 @@ define token srcDestByte2 (8) drk03_ = (0,3) # constraint bits d7_ = (7,7) + d67_ = (6,7) d57_ = (5,7) s3_ = (3,3) s13_ = (1,3) @@ -85,13 +87,13 @@ attach variables [ wrj47_d2 ] [ # NOTE: must use constraints DRK, DRKD and DRKS attach variables [ drk47 drk03 drk47_ drk03_ ] [ DR0 DR4 DR8 DR12 DR16 DR20 DR24 DR28 - DPX SPX _ _ _ _ _ _ + _ _ _ _ _ _ DPX SPX ]; -@define DRK47 "drk47 & (d7=0 | d57=4)" # constraint for using drk47 -@define DRK03 "drk03 & (s3=0 | s13=4)" # constraint for using drk03 -@define DRK47_ "drk47_ & (d7_=0 | d57_=4)" # constraint for using drk47_ - +@define DRK47 "drk47 & (d7=0 | d67=3)" # constraint for using drk47 +@define DRK03 "drk03 & (s3=0 | s23=3)" # constraint for using drk03 +@define DRK47_ "drk47_ & (d7_=0 | d67_=3)" # constraint for using drk47_ + AtWRjb: "@"^wrj47 is wrj47 { ptr:3 = zext(wrj47); export *:1 ptr; } AtWRjw: "@"^wrj47 is wrj47 { ptr:3 = zext(wrj47); export *:2 ptr; } @@ -497,7 +499,7 @@ macro pop24(val) { # MOVH DRk,#data16 -:MOVH drk47,Data16x0 is $(GROUP3) & ophi=7 & oplo=14; $(DRK47) & s03=12; Data16x0 { drk47 = (drk47 & 0xffff0000) | (Data16x0 << 16); } +:MOVH drk47,Data16x0 is $(GROUP3) & ophi=7 & oplo=0xa; $(DRK47) & s03=0xc; Data16x0 { drk47 = (drk47 & 0xffff) | (Data16x0 << 16); } # MOVS WRj,Rm :MOVZ wrj47,rm03 is $(GROUP3) & ophi=1 & oplo=10; wrj47 & rm03 { wrj47 = sext(rm03); } diff --git a/Ghidra/Processors/AARCH64/data/languages/AARCH64neon.sinc b/Ghidra/Processors/AARCH64/data/languages/AARCH64neon.sinc index af8ccb222d..4ff3386c3c 100644 --- a/Ghidra/Processors/AARCH64/data/languages/AARCH64neon.sinc +++ b/Ghidra/Processors/AARCH64/data/languages/AARCH64neon.sinc @@ -16042,7 +16042,7 @@ is b_2431=0b01011110 & b_2223=0b00 & b_2121=0 & Rm_VPR128.4S & b_1015=0b000000 & :sha1h Rd_FPR32, Rn_FPR32 is b_2431=0b01011110 & b_2223=0b00 & b_1721=0b10100 & b_1216=0b00000 & b_1011=0b10 & Rn_FPR32 & Rd_FPR32 & Zd { - Rd_FPR32 = Rn_FPR32 << 30:1; + Rd_FPR32 = Rn_FPR32 << 30:1 | (Rn_FPR32 >> 2:1); zext_zs(Zd); # zero upper 28 bytes of Zd } diff --git a/Ghidra/Processors/MIPS/data/languages/mips16.sinc b/Ghidra/Processors/MIPS/data/languages/mips16.sinc index 61175f272d..560b3406e0 100644 --- a/Ghidra/Processors/MIPS/data/languages/mips16.sinc +++ b/Ghidra/Processors/MIPS/data/languages/mips16.sinc @@ -517,7 +517,8 @@ SAVE_TOP: SAVE_ARG^EXT_FRAME^SAVE_RA^SAVE_SREG^SAVE_STAT is EXT_FRAME & SAVE_RA } -:jal Abs26_m16 is ISA_MODE=1 & RELP=1 & ext_isjal=1 & ext_tgt_x=0 & Abs26_m16 [ ext_delay=0b10; globalset(inst_next, ext_delay);] { +:jal Abs26_m16 is ISA_MODE=1 & RELP=1 & ext_isjal=1 & ext_tgt_x=0 & Abs26_m16 +[ ext_delay=0b10; globalset(inst_next, ext_delay); ] { ra = inst_next | 0x1; delayslot( 1 ); call Abs26_m16; @@ -525,7 +526,7 @@ SAVE_TOP: SAVE_ARG^EXT_FRAME^SAVE_RA^SAVE_SREG^SAVE_STAT is EXT_FRAME & SAVE_RA } :jalr m16_rx is ISA_MODE=1 & RELP=1 & ext_isjal=0 & m16_op=0b11101 & m16_rr_nd=0 & m16_rr_l=1 & m16_rr_ra=0 & m16_rr_f=0b00000 & m16_rx -[ ext_delay=0b01; globalset(inst_next, ext_delay); globalset(inst_start, ext_delay); ] { +[ ext_delay=0b01; globalset(inst_next, ext_delay); ] { JXWritePC(m16_rx); ra = inst_next | 0x1; delayslot( 1 ); @@ -539,7 +540,7 @@ SAVE_TOP: SAVE_ARG^EXT_FRAME^SAVE_RA^SAVE_SREG^SAVE_STAT is EXT_FRAME & SAVE_RA } :jalx Abs26_m16 is ISA_MODE=1 & RELP=1 & ext_isjal=1 & ext_tgt_x=1 & Abs26_m16 -[ ext_delay=0b10; ISA_MODE = 0; globalset(Abs26_m16, ISA_MODE); globalset(inst_next, ext_delay); globalset(inst_start, ext_delay); ] { +[ ext_delay=0b10; ISA_MODE = 0; globalset(Abs26_m16, ISA_MODE); globalset(inst_next, ext_delay); ] { ra = inst_next | 0x1; delayslot( 1 ); ISAModeSwitch = 0; @@ -547,14 +548,14 @@ SAVE_TOP: SAVE_ARG^EXT_FRAME^SAVE_RA^SAVE_SREG^SAVE_STAT is EXT_FRAME & SAVE_RA } :jr ra is ISA_MODE=1 & RELP=1 & ext_isjal=0 & m16_op=0b11101 & m16_rr_nd=0 & m16_rr_l=0 & m16_rr_ra=1 & ra & m16_rr_f=0b00000 & m16_rx=0 -[ ext_delay=0b01; globalset(inst_next, ext_delay); globalset(inst_start, ext_delay); ] { +[ ext_delay=0b01; globalset(inst_next, ext_delay); ] { JXWritePC(ra); delayslot( 1 ); return [pc]; } :jr m16_rx is ISA_MODE=1 & RELP=1 & ext_isjal=0 & m16_op=0b11101 & m16_rr_nd=0 & m16_rr_l=0 & m16_rr_ra=0 & m16_rr_f=0b00000 & m16_rx -[ ext_delay=0b01; globalset(inst_next, ext_delay); globalset(inst_start, ext_delay); ] { +[ ext_delay=0b01; globalset(inst_next, ext_delay); ] { JXWritePC(m16_rx); delayslot( 1 ); goto [pc]; @@ -883,7 +884,7 @@ E2_REGOFF: imm is ext_imm_2124 & m16_i_imm [ imm = m16_i_imm | (ext_imm_2124 << lockload(tmp); } -:lui m16_rx, EXT_LIU8 is ISA_MODE=1 & RELP=1 & ext_isjal=0 & m16_op=0b01101 & m16_rx & m16_ri_z=1 & EXT_LIU8 { +:lui m16_rx, EXT_LIU8 is ISA_MODE=1 & RELP=1 & ext_isjal=0 & ext_is_ext=1 & m16_op=0b01101 & m16_rx & m16_ri_z=1 & EXT_LIU8 { m16_rx = zext(EXT_LIU8) << 16; } diff --git a/Ghidra/RuntimeScripts/Common/support/gradle/gradle-wrapper.properties b/Ghidra/RuntimeScripts/Common/support/gradle/gradle-wrapper.properties index 23449a2b54..37f78a6af8 100644 --- a/Ghidra/RuntimeScripts/Common/support/gradle/gradle-wrapper.properties +++ b/Ghidra/RuntimeScripts/Common/support/gradle/gradle-wrapper.properties @@ -1,6 +1,6 @@ distributionBase=GRADLE_USER_HOME distributionPath=wrapper/dists -distributionUrl=https\://services.gradle.org/distributions/gradle-9.2.1-bin.zip +distributionUrl=https\://services.gradle.org/distributions/gradle-9.3.1-bin.zip networkTimeout=10000 validateDistributionUrl=true zipStoreBase=GRADLE_USER_HOME diff --git a/Ghidra/RuntimeScripts/Common/support/launch.properties b/Ghidra/RuntimeScripts/Common/support/launch.properties index 14f0096465..454cb7e1e5 100644 --- a/Ghidra/RuntimeScripts/Common/support/launch.properties +++ b/Ghidra/RuntimeScripts/Common/support/launch.properties @@ -51,6 +51,13 @@ VMARGS=-Djdk.tls.client.protocols=TLSv1.2,TLSv1.3 # #VMARGS=-Djavax.net.debug=ssl +# When using Java 21.0.10 or later and connecting to an older Ghidra Server (pre-12.0.3) the following +# connection error may occur. +# ... SSLHandshakeException: (certificate_unknown) No matching found +# If unable to upgrade your Ghidra Server this property setting may be uncommented to disable the +# hostname check. +#VMARGS=-Djdk.rmi.ssl.client.enableEndpointIdentification=false + # The following property will limit the number of processor cores that Ghidra # will use for thread pools. If not specified, it will use the default number # of processors returned from Runtime.getRuntime().getAvailableProcessors(). diff --git a/Ghidra/Test/DebuggerIntegrationTest/src/test/java/ghidra/app/plugin/core/debug/service/breakpoint/DebuggerRmiLogicalBreakpointServiceTest.java b/Ghidra/Test/DebuggerIntegrationTest/src/test/java/ghidra/app/plugin/core/debug/service/breakpoint/DebuggerRmiLogicalBreakpointServiceTest.java index baa65a74e3..b13e36a531 100644 --- a/Ghidra/Test/DebuggerIntegrationTest/src/test/java/ghidra/app/plugin/core/debug/service/breakpoint/DebuggerRmiLogicalBreakpointServiceTest.java +++ b/Ghidra/Test/DebuggerIntegrationTest/src/test/java/ghidra/app/plugin/core/debug/service/breakpoint/DebuggerRmiLogicalBreakpointServiceTest.java @@ -16,22 +16,31 @@ package ghidra.app.plugin.core.debug.service.breakpoint; import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotSame; import java.util.*; import org.junit.Before; +import org.junit.Test; import db.Transaction; +import generic.Unique; +import ghidra.app.plugin.core.debug.gui.breakpoint.DebuggerBreakpointsPlugin; +import ghidra.app.plugin.core.debug.gui.model.DebuggerModelPlugin; +import ghidra.app.plugin.core.debug.service.control.DebuggerControlServicePlugin; import ghidra.app.plugin.core.debug.service.modules.DebuggerStaticMappingUtils; import ghidra.app.plugin.core.debug.service.tracermi.TraceRmiTarget; +import ghidra.app.services.DebuggerControlService; +import ghidra.debug.api.breakpoint.LogicalBreakpoint; +import ghidra.debug.api.control.ControlMode; +import ghidra.debug.api.tracemgr.DebuggerCoordinates; import ghidra.program.model.address.Address; import ghidra.program.model.listing.Program; import ghidra.program.util.ProgramLocation; import ghidra.trace.database.ToyDBTraceBuilder; import ghidra.trace.model.*; +import ghidra.trace.model.breakpoint.*; import ghidra.trace.model.breakpoint.TraceBreakpointKind.TraceBreakpointKindSet; -import ghidra.trace.model.breakpoint.TraceBreakpointLocation; -import ghidra.trace.model.breakpoint.TraceBreakpointSpec; import ghidra.trace.model.memory.TraceMemoryRegion; import ghidra.trace.model.modules.TraceStaticMapping; import ghidra.trace.model.target.TraceObject; @@ -249,4 +258,203 @@ public class DebuggerRmiLogicalBreakpointServiceTest extends assertEquals(Map.ofEntries( Map.entry("breakpoint", expectedLoc.getSpecification().getObject())), args); } + + @Test + public void testAddTraceBreakpointThenModifyRange_Lone() throws Throwable { + // These are for interactive debugging + //addPlugin(tool, DebuggerModelPlugin.class); + //addPlugin(tool, DebuggerBreakpointsPlugin.class); + + DebuggerControlService controlService = + addPlugin(tool, DebuggerControlServicePlugin.class); + + createTrace(); + traceManager.openTrace(tb.trace); + traceManager.activate(DebuggerCoordinates.NOWHERE.trace(tb.trace).snap(1)); + // Needs to have a target or be emulated for the breakpoint service to care + controlService.setCurrentMode(tb.trace, ControlMode.RW_EMULATOR); + + TraceBreakpointLocation bpt; + try (Transaction tid = tb.startTransaction()) { + tb.createRootObject(SCHEMA_CTX); + bpt = tb.trace.getBreakpointManager() + .addBreakpoint("Processes[1].Breakpoints[0][0]", Lifespan.nowOn(0), + tb.addr(0x55550123), Set.of(), Set.of(TraceBreakpointKind.SW_EXECUTE), + false, ""); + } + waitForDomainObject(tb.trace); + waitOn(breakpointService.changesSettled()); + changeListener.assertAgreesWithService(); + + LogicalBreakpoint lbBefore = + Unique.assertOne(breakpointService.getBreakpointsAt(tb.trace, tb.addr(0x55550123))); + + try (Transaction tid = tb.startTransaction()) { + bpt.setRange(Lifespan.nowOn(1), tb.range(0x56660123)); + } + waitForDomainObject(tb.trace); + waitOn(breakpointService.changesSettled()); + changeListener.assertAgreesWithService(); + + LogicalBreakpoint lbAfter = + Unique.assertOne(breakpointService.getBreakpointsAt(tb.trace, tb.addr(0x56660123))); + assertNotSame(lbBefore, lbAfter); + assertEquals(Set.of(lbAfter), breakpointService.getAllBreakpoints()); + assertEquals(Set.of(bpt), lbAfter.getTraceBreakpoints()); + } + + @Test + public void testAddTraceBreakpointThenModifyRange_LoneThenNullAndBack() throws Throwable { + // These are for interactive debugging + addPlugin(tool, DebuggerModelPlugin.class); + addPlugin(tool, DebuggerBreakpointsPlugin.class); + + DebuggerControlService controlService = + addPlugin(tool, DebuggerControlServicePlugin.class); + + createTrace(); + traceManager.openTrace(tb.trace); + traceManager.activate(DebuggerCoordinates.NOWHERE.trace(tb.trace).snap(1)); + // Needs to have a target or be emulated for the breakpoint service to care + controlService.setCurrentMode(tb.trace, ControlMode.RW_EMULATOR); + + TraceBreakpointLocation bpt; + try (Transaction tid = tb.startTransaction()) { + tb.createRootObject(SCHEMA_CTX); + bpt = tb.trace.getBreakpointManager() + .addBreakpoint("Processes[1].Breakpoints[0][0]", Lifespan.nowOn(0), + tb.addr(0x55550123), Set.of(), Set.of(TraceBreakpointKind.SW_EXECUTE), + false, ""); + } + waitForDomainObject(tb.trace); + waitOn(breakpointService.changesSettled()); + changeListener.assertAgreesWithService(); + + Unique.assertOne(breakpointService.getBreakpointsAt(tb.trace, tb.addr(0x55550123))); + + try (Transaction tid = tb.startTransaction()) { + bpt.getObject() + .setAttribute(Lifespan.nowOn(1), TraceBreakpointLocation.KEY_RANGE, null); + } + waitForDomainObject(tb.trace); + waitOn(breakpointService.changesSettled()); + changeListener.assertAgreesWithService(); + + assertEquals(Set.of(), breakpointService.getAllBreakpoints()); + + try (Transaction tid = tb.startTransaction()) { + bpt.setRange(Lifespan.nowOn(1), tb.range(0x56660123)); + } + waitForDomainObject(tb.trace); + waitOn(breakpointService.changesSettled()); + changeListener.assertAgreesWithService(); + + LogicalBreakpoint lbAfter = + Unique.assertOne(breakpointService.getBreakpointsAt(tb.trace, tb.addr(0x56660123))); + assertEquals(Set.of(lbAfter), breakpointService.getAllBreakpoints()); + assertEquals(Set.of(bpt), lbAfter.getTraceBreakpoints()); + } + + @Test + public void testAddTraceBreakpointThenModifyRange_MappedThenLone() throws Throwable { + // These are for interactive debugging + //addPlugin(tool, DebuggerModelPlugin.class); + //addPlugin(tool, DebuggerBreakpointsPlugin.class); + + DebuggerControlService controlService = + addPlugin(tool, DebuggerControlServicePlugin.class); + + createTrace(); + traceManager.openTrace(tb.trace); + traceManager.activate(DebuggerCoordinates.NOWHERE.trace(tb.trace).snap(1)); + // Needs to have a target or be emulated for the breakpoint service to care + controlService.setCurrentMode(tb.trace, ControlMode.RW_EMULATOR); + + createProgramFromTrace(); + intoProject(program); + programManager.openProgram(program); + + TraceBreakpointLocation bpt; + try (Transaction tid = tb.startTransaction()) { + tb.createRootObject(SCHEMA_CTX); + bpt = tb.trace.getBreakpointManager() + .addBreakpoint("Processes[1].Breakpoints[0][0]", Lifespan.nowOn(0), + tb.addr(0x55550123), Set.of(), Set.of(TraceBreakpointKind.SW_EXECUTE), + false, ""); + addTextMappingDead(0, program, tb); + } + waitForDomainObject(tb.trace); + waitOn(mappingService.changesSettled()); + waitOn(breakpointService.changesSettled()); + changeListener.assertAgreesWithService(); + + LogicalBreakpoint lbBefore = + Unique.assertOne(breakpointService.getBreakpointsAt(tb.trace, tb.addr(0x55550123))); + + try (Transaction tid = tb.startTransaction()) { + bpt.setRange(Lifespan.nowOn(1), tb.range(0x56660123)); + // NOTE: New address is not mapped + } + waitForDomainObject(tb.trace); + waitOn(breakpointService.changesSettled()); + changeListener.assertAgreesWithService(); + + LogicalBreakpoint lbAfter = + Unique.assertOne(breakpointService.getBreakpointsAt(tb.trace, tb.addr(0x56660123))); + assertNotSame(lbBefore, lbAfter); + assertEquals(Set.of(lbBefore, lbAfter), breakpointService.getAllBreakpoints()); + assertEquals(Set.of(), lbBefore.getTraceBreakpoints()); + assertEquals(Set.of(bpt), lbAfter.getTraceBreakpoints()); + } + + @Test + public void testAddTraceBreakpointThenModifyRange_Mapped() throws Throwable { + // These are for interactive debugging + //addPlugin(tool, DebuggerModelPlugin.class); + //addPlugin(tool, DebuggerBreakpointsPlugin.class); + + DebuggerControlService controlService = + addPlugin(tool, DebuggerControlServicePlugin.class); + + createTrace(); + traceManager.openTrace(tb.trace); + traceManager.activate(DebuggerCoordinates.NOWHERE.trace(tb.trace).snap(1)); + // Needs to have a target or be emulated for the breakpoint service to care + controlService.setCurrentMode(tb.trace, ControlMode.RW_EMULATOR); + + createProgramFromTrace(); + intoProject(program); + programManager.openProgram(program); + + TraceBreakpointLocation bpt; + try (Transaction tid = tb.startTransaction()) { + tb.createRootObject(SCHEMA_CTX); + bpt = tb.trace.getBreakpointManager() + .addBreakpoint("Processes[1].Breakpoints[0][0]", Lifespan.nowOn(0), + tb.addr(0x55550123), Set.of(), Set.of(TraceBreakpointKind.SW_EXECUTE), + false, ""); + addTextMappingDead(0, program, tb); + } + waitForDomainObject(tb.trace); + waitOn(mappingService.changesSettled()); + waitOn(breakpointService.changesSettled()); + changeListener.assertAgreesWithService(); + + LogicalBreakpoint lbBefore = + Unique.assertOne(breakpointService.getBreakpointsAt(tb.trace, tb.addr(0x55550123))); + + try (Transaction tid = tb.startTransaction()) { + bpt.setRange(Lifespan.nowOn(1), tb.range(0x55550124)); + } + waitForDomainObject(tb.trace); + waitOn(breakpointService.changesSettled()); + changeListener.assertAgreesWithService(); + + LogicalBreakpoint lbAfter = + Unique.assertOne(breakpointService.getBreakpointsAt(tb.trace, tb.addr(0x55550124))); + assertNotSame(lbBefore, lbAfter); + assertEquals(Set.of(lbBefore, lbAfter), breakpointService.getAllBreakpoints()); + assertEquals(Set.of(), lbBefore.getTraceBreakpoints()); + assertEquals(Set.of(bpt), lbAfter.getTraceBreakpoints()); + } } diff --git a/Ghidra/Test/DebuggerIntegrationTest/src/test/java/ghidra/pcode/exec/TraceRmiPcodeExecTest.java b/Ghidra/Test/DebuggerIntegrationTest/src/test/java/ghidra/pcode/exec/TraceRmiPcodeExecTest.java index 187a712a0d..9d2f585d30 100644 --- a/Ghidra/Test/DebuggerIntegrationTest/src/test/java/ghidra/pcode/exec/TraceRmiPcodeExecTest.java +++ b/Ghidra/Test/DebuggerIntegrationTest/src/test/java/ghidra/pcode/exec/TraceRmiPcodeExecTest.java @@ -114,12 +114,6 @@ public class TraceRmiPcodeExecTest extends AbstractGhidraHeadedDebuggerIntegrati return null; }); - /** - * TODO: This second handle should not be necessary. The KNOWN ought to carry into the - * scratch snapshot. - */ - handleReadRegsInvocation(objRegs, () -> null); - byte[] result = waitOn(futResult); assertEquals(new BigInteger("11"), executor.getArithmetic().toBigInteger(result, Purpose.INSPECT)); diff --git a/Ghidra/Test/IntegrationTest/src/test.slow/java/ghidra/server/remote/ServerTestUtil.java b/Ghidra/Test/IntegrationTest/src/test.slow/java/ghidra/server/remote/ServerTestUtil.java index 2728663074..5aebdaa0d7 100644 --- a/Ghidra/Test/IntegrationTest/src/test.slow/java/ghidra/server/remote/ServerTestUtil.java +++ b/Ghidra/Test/IntegrationTest/src/test.slow/java/ghidra/server/remote/ServerTestUtil.java @@ -20,7 +20,7 @@ import java.net.*; import java.rmi.registry.LocateRegistry; import java.rmi.registry.Registry; import java.security.KeyStore.PrivateKeyEntry; -import java.util.ArrayList; +import java.util.*; import java.util.function.Consumer; import java.util.zip.ZipEntry; import java.util.zip.ZipInputStream; @@ -955,7 +955,25 @@ public class ServerTestUtil { TEST_PKI_SERVER_PASSPHRASE + "): " + serverKeystorePath); PKIUtils.createKeyEntry("test-sig", TEST_PKI_SERVER_DN, 2, caEntry, serverKeystoreFile, - "PKCS12", null, TEST_PKI_SERVER_PASSPHRASE.toCharArray()); + "PKCS12", getLocalHostnames(), TEST_PKI_SERVER_PASSPHRASE.toCharArray()); + } + + private static Collection getLocalHostnames() throws SocketException { + + // Collect alternate hostnames for inclusion in certificate + Set altNames = new TreeSet<>(); + Enumeration nets = NetworkInterface.getNetworkInterfaces(); + while (nets.hasMoreElements()) { + NetworkInterface netint = nets.nextElement(); + Enumeration addrs = netint.getInetAddresses(); + while (addrs.hasMoreElements()) { + InetAddress addr = addrs.nextElement(); + altNames.add(addr.getHostAddress()); + altNames.add(addr.getHostName()); + altNames.add(addr.getCanonicalHostName()); + } + } + return altNames; } /** diff --git a/Ghidra/application.properties b/Ghidra/application.properties index 1e001bce10..9e1f98909f 100644 --- a/Ghidra/application.properties +++ b/Ghidra/application.properties @@ -1,5 +1,5 @@ application.name=Ghidra -application.version=12.0.2 +application.version=12.0.3 application.release.name=DEV application.layout.version=3 application.gradle.min=8.5