eliott/ghidra
1
0
Fork 0
mirror of https://github.com/NationalSecurityAgency/ghidra.git synced 2026-04-25 17:25:17 +02:00
Code Issues Packages Projects Releases Wiki Activity

GP-6643 Added ability to order Libraries via an ordinal assignment. This ordered list is now used by external symbol resolver analyzer.

Browse Source
This commit is contained in:
ghidra1
2026-04-20 09:27:11 -04:00
parent 7be1d8c943
commit 5e82e2869c
32 changed files with 1473 additions and 553 deletions
Download Patch File Download Diff File Expand all files Collapse all files

137
Ghidra/Features/Base/src/main/help/help/topics/ReferencesPlugin/external_program_names.htm
View File

@@ -14,14 +14,36 @@
<BODY lang="EN-US">
<H1><A name="View_External_Program_Names"></A>External Program Names</H1>
<P>An external reference is a reference to a location in another program. The reference
destination includes the name of some program. To use an external reference to navigate, the
external program name must be associated with an existing program file in the Ghidra project.
<A name="Resolve"></A>If the association has been defined, then the external reference is said
to have been <B><I>resolved</I></B>. The <I>External Programs</I> view manages the associations
between external program names and program files. The table shows all external program names
and their associated links to program files. Use the <I>External Programs</I> view to add
external names, delete external names, set associations, and clear associations.</P>
<P>An external location identifies an external function or data dependency
associated with a named Library (i.e., External Program). One or more external references may refer to
a single external location. Thunk functions within a Program may also refer to an external
location if it corresponds to a function. Each Library defined within Ghidra can optionally
be associated with another Program file within the Ghidra project. </P>
<P>There is a reserved
Library named <B>&lt;EXTERNAL&gt;</B> which is a holding area for external locations whose
associated Library is unknown (commonly used by ELF Imports).
The <B>External Symbol Resolver</B> analyzer can be used to search
for these external locations among the ordered list of Libraries which have a Program file association.
The ordered Library sequence dictates the order that this analyzer searches through them.</P>
<P>To navigate on an external reference/location, or resolve external locations, the
external Library name must be associated with an existing Program file in the same project.
<A name="Resolve"></A>If a Library's Program association has been specified, then any related
external reference is said to have been <B><I>resolved</I></B>. </P>
<P>The <I>External Programs</I> view manages the associations
between external Library names and Program files as well as the ordered Library sequence.
Use the <I>External Programs</I> view to add/delete external Library names,
set/clear Program associations and adjust search order using up/down placement actions.
Other than the up/down ordering functionality, the
<A href="../SymbolTreePlugin/SymbolTree.htm#Externals">Symbol Tree</A> Imports node provides
similar actions plus the ability to navigate.</P>
<P><I><IMG src="help/shared/note.png" border="0"> The term "External Program" within Ghidra
is used interchangeably with the term "Library" and represents a dependency. Each defined
Library has a corresponding Library Symbol within a program's symbol table and may have related
external locations.</P>
<H2><A name="ExternalNamesDialog"></A>External Programs View</H2>
@@ -29,26 +51,33 @@
</P>
<BLOCKQUOTE>
<P>The <I>External Programs</I> view consists of a main scrollable list of external program
<P>The <I>External Programs</I> view consists of a main scrollable list of external Library
names and their associated Ghidra program files.</P>
<BLOCKQUOTE>
<H3>Name Column</H3>
<BLOCKQUOTE>
<P>The name of the external program. Many external programs will share the same external
program name. Setting or changing the associated Ghidra file will affect all the external
references to that name. Double-click on this field to edit the name. After you change
the name, hit the &lt;Enter&gt; key.</P>
<P>The name of the external Library. Many external function and/or data locations will
correspond to the same external Library within a Ghidra program which are considered to
be Imports.</P>
<P>The name used within the current program to identify the Library may be
changed. To do this, double-click on the Name to enter edit mode. After you change
the name, press &lt;Enter&gt; to commit the change.</P>
</BLOCKQUOTE>
<H3>Ghidra Program Column<BR>
</H3>
<BLOCKQUOTE>
<P>The Ghidra file associated with the external program name. This field is blank if
external reference has not been resolved. Ghidra will not be able to "follow" an external
reference if its external program name does not have an associated Ghidra file.</P>
<P>The Ghidra program file associated with the external Library name. This field is blank if
the external Library has not been resolved. Ghidra will not be able to "follow" an external
location reference into a Library, or search for external symbols within a Library,
if a Library does not have a Ghidra program file association.</P>
<P>See <A href="#ChooseExternalProgram">Set External Path Association</A> for changing the
path shown.</P>
</BLOCKQUOTE>
<H3><A name="Add_External_Program_Name"></A>Add Button</H3>
@@ -58,14 +87,48 @@
for entering a new external program name.<BR>
</P>
</BLOCKQUOTE>
<H3><A name="Delete_External_Program_Name"></A>Delete External Name Button</H3>
<BLOCKQUOTE>
<P>The <SPAN style="font-weight: bold;">Delete</SPAN> <IMG alt="" src=
"images/edit-delete.png"> button deletes the selected external Library names from the
program.&nbsp; If a selected external Library name has associated external locations, it can
not be deleted. The <SPAN style="font-weight: bold;">Delete</SPAN> button is enabled
whenever one or more rows are selected.</P>
</BLOCKQUOTE>
<H3><A name="Move_Library_Up"></A>Move Library Up Button</H3>
<BLOCKQUOTE>
<P>The <SPAN style="font-weight: bold;">Up</SPAN> <IMG alt="" src=
"images/up.png"> button will shift a selected external Library up
within the list of Libraries thus increasing its priority when used to
search for external symbols. The <SPAN style="font-weight: bold;">Up</SPAN>
button is enabled whenever one external program name is selected and can be moved
up within the Library list.</P>
<P>
</BLOCKQUOTE>
<H3><A name="Move_Library_Down"></A>Move Library Down Button</H3>
<BLOCKQUOTE>
<P>The <SPAN style="font-weight: bold;">Down</SPAN> <IMG alt="" src=
"images/down.png"> button will shift a selected external Library down
within the list of Libraries thus reducing its priority when used to
search for external symbols. The <SPAN style="font-weight: bold;">Down</SPAN>
button is enabled whenever one external program name is selected and can be moved
down within the Library list.</P>
<P>
</BLOCKQUOTE>
<H3><A name="ChooseExternalProgram"></A><A name="Set_External_Name_Association"></A>Set
External Name Association Button</H3>
External Path Association Button</H3>
<BLOCKQUOTE>
<P>The <SPAN style="font-weight: bold;">Set</SPAN> <IMG alt="" src=
"images/editbytes.gif"> button brings up a <I>Ghidra program chooser</I> dialog. Choose a
Ghidra program file to associate it with the selected external program name. This button
Ghidra program file to associate it with the selected external Library name. This button
is only enabled when a single external program name is selected.</P>
</BLOCKQUOTE>
</BLOCKQUOTE>
@@ -77,26 +140,13 @@
<BLOCKQUOTE>
<BLOCKQUOTE>
<H3><A name="Clear_External_Name_Association"></A>Clear External Name Association
<H3><A name="Clear_External_Name_Association"></A>Clear External Path Association
Button</H3>
<BLOCKQUOTE>
<P>This <SPAN style="font-weight: bold;">Clear <IMG alt="" src=
"images/edit-delete.png">&nbsp;</SPAN> button clears the assocated program for all the
selected external program names.<BR>
</P>
</BLOCKQUOTE>
<H3><A name="Delete_External_Program_Name"></A>Delete External Name Button</H3>
<BLOCKQUOTE>
<P>This <SPAN style="font-weight: bold;">Delete</SPAN> <IMG alt="" src=
"images/edit-delete.png"> button deletes the selected external program names from the
program.&nbsp; If a selected external program name contains external locations, it can
not be deleted. The <SPAN style="font-weight: bold;">Delete</SPAN> button is enabled
whenever one or more external program names are selected.</P>
<P><BR>
<P>The <SPAN style="font-weight: bold;">Clear <IMG alt="" src=
"Icons.CLEAR_ICON">&nbsp;</SPAN> button clears the associated program path for all the
selected rows.<BR>
</P>
</BLOCKQUOTE>
</BLOCKQUOTE>
@@ -115,17 +165,12 @@
<LI>Enter the new external program name into the pop-up dialog.</LI>
<LI style="list-style: none">
&lt;&gt;
<P><I><IMG src="help/shared/note.png" border="0">Any new External Program added
will be placed at the bottom of the list. If relying on external symbol resolution
analysis and the library search order is important, the new entry may be moved up
in the list using the Up button.
</I></P>
<P><I><IMG src="help/shared/note.png" border="0"> If the table is sorted by Name, then
the name you enter will be placed at the correct position in the table to maintain the
sort order. The sort icon <IMG src="Icons.SORT_ASCENDING_ICON" border="0"> or <IMG
src="Icons.SORT_DESCENDING_ICON" border="0"> indicates the order and what column is
being sorted. You can also sort by Ghidra Pathname by clicking on the header for this
column. Click on the sort icon to change the order.</I></P>
&lt;/&gt;
</LI>
</OL>
<H3>Resolving an External Name to an existing Ghidra program<BR>
@@ -157,7 +202,7 @@
<LI>Click on the external program name that has an association to be cleared.</LI>
<LI>Press the <B>Clear</B> <IMG alt="" src="images/erase16.png"> button</LI>
<LI>Press the <B>Clear</B> <IMG alt="" src="Icons.CLEAR_ICON"> button</LI>
</OL>
<H3>Removing an External Program Name</H3>

BIN
Ghidra/Features/Base/src/main/help/help/topics/ReferencesPlugin/images/External_names_dialog.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 8.9 KiB

After

Width:  |  Height:  |  Size: 11 KiB

33
Ghidra/Features/Base/src/main/java/ghidra/app/cmd/refs/ClearExternalNameCmd.java → Ghidra/Features/Base/src/main/java/ghidra/app/cmd/refs/ClearExternalPathCmd.java
View File

@@ -4,9 +4,9 @@
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
*
* http://www.apache.org/licenses/LICENSE-2.0
*
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -16,37 +16,46 @@
package ghidra.app.cmd.refs;
import ghidra.framework.cmd.Command;
import ghidra.program.model.listing.Library;
import ghidra.program.model.listing.Program;
import ghidra.util.exception.AssertException;
import ghidra.program.model.symbol.ExternalManager;
import ghidra.util.exception.InvalidInputException;
/**
* Command to remove an external program name from the reference manager.
* Command to clear the external program path associated with an external Library.
*
*/
public class ClearExternalNameCmd implements Command<Program> {
public class ClearExternalPathCmd implements Command<Program> {
private String externalName;
private String status;
private boolean userDefined = true;
/**
* Constructs a new command removing an external program name.
* @param externalName the name of the external program name to be removed.
* Constructs a new command for clearing the external program path associated with a
* specified external Library.
* @param externalName external Library name
*/
public ClearExternalNameCmd(String externalName) {
public ClearExternalPathCmd(String externalName) {
this.externalName = externalName;
}
@Override
public boolean applyTo(Program program) {
try {
program.getExternalManager().setExternalPath(externalName, null, userDefined);
// Avoid creating the Library if it does not already exist
ExternalManager externalManager = program.getExternalManager();
Library lib = externalManager.getExternalLibrary(externalName);
if (lib != null) {
externalManager.setExternalPath(externalName, null, userDefined);
return true;
}
status = "Library not found: " + externalName;
}
catch (InvalidInputException e) {
throw new AssertException(e);
status = e.getMessage();
}
return true;
return false;
}
@Override
@@ -56,7 +65,7 @@ public class ClearExternalNameCmd implements Command<Program> {
@Override
public String getName() {
return "Remove External Program Name";
return "Clear External Library Path";
}
}

47
Ghidra/Features/Base/src/main/java/ghidra/app/cmd/refs/SetExternalNameCmd.java
View File

@@ -4,9 +4,9 @@
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
*
* http://www.apache.org/licenses/LICENSE-2.0
*
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -16,7 +16,11 @@
package ghidra.app.cmd.refs;
import ghidra.framework.cmd.Command;
import ghidra.program.model.listing.Library;
import ghidra.program.model.listing.Program;
import ghidra.program.model.symbol.ExternalManager;
import ghidra.program.model.symbol.SourceType;
import ghidra.util.exception.DuplicateNameException;
import ghidra.util.exception.InvalidInputException;
/**
@@ -26,29 +30,48 @@ public class SetExternalNameCmd implements Command<Program> {
private String externalName;
private String externalPath;
private SourceType source;
private String status;
private boolean userDefined = true;
/**
* Constructs a new command for setting the external program name and path.
* @param externalName the name of the link.
* @param externalPath the path of the file to associate with this link.
* Constructs a new command for creating a Library, if it does not exist, and optionally
* setting the associated external program path. If created, a {@link SourceType#USER_DEFINED}
* source will be specified.
* @param externalName the Library name.
* @param externalPath the project file path of the program file to associate with the Library.
*/
public SetExternalNameCmd(String externalName, String externalPath) {
this(externalName, externalPath, SourceType.USER_DEFINED);
}
/**
* Constructs a new command for creating a Library, if it does not exist, and optionally
* setting the associated external program path.
* @param externalName the Library name.
* @param externalPath the project file path of the program file to associate with the Library.
* @param source the symbol source type to be applied if the library must be created.
*/
public SetExternalNameCmd(String externalName, String externalPath, SourceType source) {
this.externalName = externalName;
this.externalPath = externalPath;
this.source = source;
}
@Override
public boolean applyTo(Program program) {
try {
program.getExternalManager().setExternalPath(externalName, externalPath, userDefined);
ExternalManager externalManager = program.getExternalManager();
Library library = externalManager.getExternalLibrary(externalName);
if (library == null) {
externalManager.addExternalLibraryName(externalName, source);
}
library.setAssociatedProgramPath(externalPath);
}
catch (InvalidInputException e) {
status = "Invalid name specified";
return false;
catch (DuplicateNameException | InvalidInputException e) {
status = e.getMessage();
}
return true;
return false;
}
@Override
@@ -58,7 +81,7 @@ public class SetExternalNameCmd implements Command<Program> {
@Override
public String getName() {
return "Set External Program Name";
return "Set External Library Name and Path";
}
}

50
Ghidra/Features/Base/src/main/java/ghidra/app/merge/listing/ExternalProgramMerger.java
View File

@@ -26,8 +26,7 @@ import ghidra.app.merge.*;
import ghidra.app.merge.util.ConflictUtility;
import ghidra.app.util.HelpTopics;
import ghidra.program.model.address.Address;
import ghidra.program.model.listing.Program;
import ghidra.program.model.listing.ProgramChangeSet;
import ghidra.program.model.listing.*;
import ghidra.program.model.symbol.*;
import ghidra.program.util.ProgramMerge;
import ghidra.program.util.SimpleDiffUtility;
@@ -448,20 +447,36 @@ public class ExternalProgramMerger implements MergeResolver, ListingMergeConstan
mergeManager.updateProgress(progress,
"Merging external program information for " + name + "...");
String originalPath =
(originalName != null) ? originalExtMgr.getExternalLibraryPath(originalName) : null;
String latestPath =
(latestName != null) ? latestExtMgr.getExternalLibraryPath(latestName) : null;
String myPath = (myName != null) ? myExtMgr.getExternalLibraryPath(myName) : null;
if (same(latestName, myName) && same(latestPath, myPath)) {
Library originalLib =
(originalName != null) ? originalExtMgr.getExternalLibrary(originalName) : null;
Library latestLib =
(latestName != null) ? latestExtMgr.getExternalLibrary(latestName) : null;
Library myLib = (myName != null) ? myExtMgr.getExternalLibrary(myName) : null;
String originalPath = (originalLib != null) ? originalLib.getAssociatedProgramPath() : null;
String latestPath = (latestLib != null) ? latestLib.getAssociatedProgramPath() : null;
String myPath = (myLib != null) ? myLib.getAssociatedProgramPath() : null;
int originalOrdinal =
originalLib != null ? originalExtMgr.getLibraryOrdinal(originalName) : -1;
int latestOrdinal = latestLib != null ? latestExtMgr.getLibraryOrdinal(latestName) : -1;
int myOrdinal = myLib != null ? myExtMgr.getLibraryOrdinal(myName) : -1;
if (same(latestName, myName) && same(latestPath, myPath) && latestOrdinal == myOrdinal) {
return;
}
boolean changedLatestName = !same(originalName, latestName);
boolean changedMyName = !same(originalName, myName);
boolean changedLatestPath = !same(originalPath, latestPath);
boolean changedMyPath = !same(originalPath, myPath);
boolean changedLatest = changedLatestName || changedLatestPath;
boolean changedMy = changedMyName || changedMyPath;
// Temper ordinal changes - don't allow them to drive a conflict but try to preserve my change.
// Ordinal changes may be lost during any library change conflict resolution.
boolean changedMyOrdinal = myLib != null && myOrdinal != originalOrdinal;
boolean changedLatest = changedLatestName || changedLatestPath; // ignore ordinal change
boolean changedMy = changedMyName || changedMyPath || changedMyOrdinal;
if (changedLatest) {
if (changedMy) {
// conflict: Ask to keep latest or my
@@ -473,7 +488,7 @@ public class ExternalProgramMerger implements MergeResolver, ListingMergeConstan
if (resultID != -1 && resultName == null) {
resultName = latestName; // Need to create Library symbol in Result program.
}
autoMergeWhenOnlyLatestChanged(resultName, latestName, latestPath);
autoMergeWhenOnlyLatestChanged(resultName, latestName, latestPath, latestOrdinal);
}
}
else {
@@ -491,13 +506,13 @@ public class ExternalProgramMerger implements MergeResolver, ListingMergeConstan
symbol.getSymbolType());
}
}
autoMergeWhenOnlyMyChanged(resultName, myName, myPath);
autoMergeWhenOnlyMyChanged(resultName, myName, myPath, myOrdinal);
}
}
}
private void autoMergeWhenOnlyLatestChanged(String resultName, String latestName,
String latestPath) {
String latestPath, int latestOrdinal) {
if (resultName == null) {
// latestName appears to have been discarded during SymbolMerge.
return;
@@ -509,6 +524,9 @@ public class ExternalProgramMerger implements MergeResolver, ListingMergeConstan
else {
resultExtMgr.setExternalPath(resultName, latestPath,
isExternalUserDefined(latestPgm, latestName));
if (latestOrdinal >= 0) {
resultExtMgr.setLibraryOrdinal(resultName, latestOrdinal);
}
}
}
catch (InvalidInputException e) {
@@ -516,7 +534,8 @@ public class ExternalProgramMerger implements MergeResolver, ListingMergeConstan
}
}
private void autoMergeWhenOnlyMyChanged(String resultName, String myName, String myPath) {
private void autoMergeWhenOnlyMyChanged(String resultName, String myName, String myPath,
int myOrdinal) {
if (resultName == null) {
// myName appears to have been discarded during SymbolMerge.
return;
@@ -528,6 +547,9 @@ public class ExternalProgramMerger implements MergeResolver, ListingMergeConstan
else {
resultExtMgr.setExternalPath(resultName, myPath,
isExternalUserDefined(myPgm, myName));
if (myOrdinal >= 0) {
resultExtMgr.setLibraryOrdinal(resultName, myOrdinal);
}
}
}
catch (InvalidInputException e) {

6
Ghidra/Features/Base/src/main/java/ghidra/app/merge/listing/SymbolMerger.java
View File

@@ -4,9 +4,9 @@
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
*
* http://www.apache.org/licenses/LICENSE-2.0
*
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -2463,6 +2463,8 @@ class SymbolMerger extends AbstractListingMerger {
ExternalManager srcExtMgr = srcPgm.getExternalManager();
String path = srcExtMgr.getExternalLibraryPath(name);
// NOTE: No attempt is made to preserve library ordinal
ExternalManagerDB extMgr = (ExternalManagerDB) resultPgm.getExternalManager();
extMgr.setExternalPath(name, path, (source == SourceType.USER_DEFINED));
symbol = resultSymTab.getLibrarySymbol(name);

90
Ghidra/Features/Base/src/main/java/ghidra/app/plugin/core/references/ExternalReferencesProvider.java
View File

@@ -27,7 +27,7 @@ import docking.ActionContext;
import docking.DefaultActionContext;
import docking.action.builder.ActionBuilder;
import docking.widgets.dialogs.InputDialog;
import docking.widgets.table.AbstractSortedTableModel;
import docking.widgets.table.AbstractGTableModel;
import generic.theme.GIcon;
import ghidra.app.cmd.refs.*;
import ghidra.framework.cmd.Command;
@@ -51,6 +51,8 @@ import resources.Icons;
public class ExternalReferencesProvider extends ComponentProviderAdapter {
private static Icon ADD_ICON = Icons.ADD_ICON;
private static Icon DELETE_ICON = Icons.DELETE_ICON;
private static Icon UP_ICON = Icons.UP_ICON;
private static Icon DOWN_ICON = Icons.DOWN_ICON;
private static Icon EDIT_ICON = new GIcon("icon.base.edit.bytes");
private static Icon CLEAR_ICON = Icons.CLEAR_ICON;
@@ -98,6 +100,22 @@ public class ExternalReferencesProvider extends ComponentProviderAdapter {
.onAction(ac -> deleteExternalProgram())
.buildAndInstallLocal(this);
new ActionBuilder("Move Library Up", getOwner())
.popupMenuPath("Move Library Up")
.popupMenuIcon(UP_ICON)
.toolBarIcon(UP_ICON)
.enabledWhen(ac -> canDecrementSelectedLibraryOrdinal())
.onAction(ac -> adjustLibraryOrdinal(true))
.buildAndInstallLocal(this);
new ActionBuilder("Move Library Down", getOwner())
.popupMenuPath("Move Library Down")
.popupMenuIcon(DOWN_ICON)
.toolBarIcon(DOWN_ICON)
.enabledWhen(ac -> canIncrementSelectedLibraryOrdinal())
.onAction(ac -> adjustLibraryOrdinal(false))
.buildAndInstallLocal(this);
new ActionBuilder("Set External Name Association", getOwner())
.popupMenuPath("Set External Name Association")
.popupMenuIcon(EDIT_ICON)
@@ -238,6 +256,48 @@ public class ExternalReferencesProvider extends ComponentProviderAdapter {
}
}
private void adjustLibraryOrdinal(boolean moveUp) {
if ((moveUp && !canDecrementSelectedLibraryOrdinal()) ||
(!moveUp && !canIncrementSelectedLibraryOrdinal())) {
return;
}
int ordinalAdjustment = moveUp ? -1 : 1;
List<String> selectedExternalNames = getSelectedExternalNames();
String externalName = selectedExternalNames.get(0); // must be exactly one for us to be enabled.
ExternalManager externalManager = program.getExternalManager();
Library externalLibrary = externalManager.getExternalLibrary(externalName);
if (externalLibrary != null) {
program.withTransaction("Adjust Library Order", () -> {
int ordinal = externalManager.getLibraryOrdinal(externalName);
if (ordinal < 0) {
Msg.showError(this, mainPanel, "Library Ordering Change Error",
"Failed to update Library ordinal for: " + externalName);
return;
}
externalManager.setLibraryOrdinal(externalName, ordinal + ordinalAdjustment);
});
}
}
private boolean canDecrementSelectedLibraryOrdinal() {
int[] selectedRows = table.getSelectedRows();
if (selectedRows.length == 1 && selectedRows[0] != 0) {
ExternalNamesRow rowBefore = tableModel.getRowObject(selectedRows[0] - 1);
return !Library.UNKNOWN.equals(rowBefore.getName()); // cannot displace UNKNOWN Library
}
return false;
}
private boolean canIncrementSelectedLibraryOrdinal() {
int lastRow = table.getRowCount() - 1;
int[] selectedRows = table.getSelectedRows();
if (selectedRows.length == 1 && selectedRows[0] != lastRow) {
ExternalNamesRow row = tableModel.getRowObject(selectedRows[0]);
return !Library.UNKNOWN.equals(row.getName()); // cannot alter UNKNOWN Library ordinal
}
return false;
}
private void setExternalProgramAssociation() {
List<String> selectedExternalNames = getSelectedExternalNames();
String externalName = selectedExternalNames.get(0); // must be exactly one for us to be enabled.
@@ -268,7 +328,7 @@ public class ExternalReferencesProvider extends ComponentProviderAdapter {
private void clearExternalAssociation() {
CompoundCmd<Program> cmd = new CompoundCmd<>("Clear External Program Associations");
for (String externalName : getSelectedExternalNames()) {
cmd.add(new ClearExternalNameCmd(externalName));
cmd.add(new ClearExternalPathCmd(externalName));
}
getTool().execute(cmd, program);
}
@@ -325,7 +385,7 @@ public class ExternalReferencesProvider extends ComponentProviderAdapter {
}
class ExternalNamesTableModel extends AbstractSortedTableModel<ExternalNamesRow> {
class ExternalNamesTableModel extends AbstractGTableModel<ExternalNamesRow> {
final static int NAME_COL = 0;
final static int PATH_COL = 1;
@@ -336,18 +396,18 @@ public class ExternalReferencesProvider extends ComponentProviderAdapter {
private List<ExternalNamesRow> paths = new ArrayList<>();
void updateTableData() {
paths.clear();
if (program != null && isVisible()) {
ExternalManager extMgr = program.getExternalManager();
// NOTE: Keep table in ordinal sequence as provided by external manager
String[] programNames = extMgr.getExternalLibraryNames();
Arrays.sort(programNames);
for (String programName : programNames) {
if (Library.UNKNOWN.equals(programName)) {
continue;
}
ExternalNamesRow path = new ExternalNamesRow(programName,
extMgr.getExternalLibraryPath(programName));
paths.add(path);
@@ -391,11 +451,6 @@ public class ExternalReferencesProvider extends ComponentProviderAdapter {
return "External Programs Model";
}
@Override
public boolean isSortable(int columnIndex) {
return true;
}
@Override
public boolean isCellEditable(int rowIndex, int columnIndex) {
if (columnIndex == NAME_COL) {
@@ -448,21 +503,6 @@ public class ExternalReferencesProvider extends ComponentProviderAdapter {
}
}
@Override
protected Comparator<ExternalNamesRow> createSortComparator(int columnIndex) {
if (columnIndex == PATH_COL) {
// force the path column to have a secondary compare using the name column
// to ensure a 'stable' sort. Without this during analysis
// the constant updates cause the table to sort randomly when
// there are lots of empty path values.
Comparator<ExternalNamesRow> c1 =
(r1, r2) -> Objects.requireNonNullElse(r1.getPath(), "")
.compareTo(Objects.requireNonNullElse(r2.getPath(), ""));
return c1.thenComparing((r1, r2) -> r1.getName().compareTo(r2.getName()));
}
return super.createSortComparator(columnIndex);
}
}
}

3
Ghidra/Features/Base/src/main/java/ghidra/app/plugin/core/symboltree/SymbolTreeProvider.java
View File

@@ -645,8 +645,7 @@ public class SymbolTreeProvider extends ComponentProviderAdapter {
.each(SYMBOL_REMOVED).call(this::processSymbolRemoved)
.each(EXTERNAL_ENTRY_ADDED, EXTERNAL_ENTRY_REMOVED)
.call(this::processExternalEntryChanged)
.any(EXTERNAL_PATH_CHANGED, EXTERNAL_NAME_ADDED,
EXTERNAL_NAME_REMOVED, EXTERNAL_NAME_CHANGED)
.any(EXTERNAL_NAME_ADDED, EXTERNAL_NAME_REMOVED, EXTERNAL_NAME_CHANGED)
// Rather than try to find each affected symbol, it is easier to just reload
// the tree. This is infrequent enough that it should not be disruptive.
.call(this::reloadTree)

12
Ghidra/Features/Base/src/main/java/ghidra/app/util/opinion/AbstractLibrarySupportLoader.java
View File

@@ -39,6 +39,7 @@ import ghidra.program.model.listing.Library;
import ghidra.program.model.listing.Program;
import ghidra.program.model.symbol.ExternalManager;
import ghidra.util.Msg;
import ghidra.util.StringUtilities;
import ghidra.util.exception.CancelledException;
import ghidra.util.exception.InvalidInputException;
import ghidra.util.task.TaskMonitor;
@@ -101,6 +102,17 @@ public abstract class AbstractLibrarySupportLoader extends AbstractProgramLoader
public static final String LOAD_ONLY_LIBRARIES_OPTION_NAME = "Only Load Libraries"; // hidden
static final boolean LOAD_ONLY_LIBRARIES_OPTION_DEFAULT = false;
/**
* Gets a program property name to represent the ordered required library of the given index
*
* @param libraryIndex The index of the required library
* @return A program property name to represent the ordered required library of the given index
*/
public static String getRequiredLibraryProperty(int libraryIndex) {
return String.format("%s %s]", "Required Library [",
StringUtilities.pad("" + libraryIndex, ' ', 4));
}
/**
* Loads bytes in a particular format into the given {@link Program}.
*

4
Ghidra/Features/Base/src/main/java/ghidra/app/util/opinion/ElfProgramBuilder.java
View File

@@ -54,7 +54,6 @@ import ghidra.program.model.scalar.Scalar;
import ghidra.program.model.symbol.*;
import ghidra.program.model.util.AddressSetPropertyMap;
import ghidra.program.model.util.CodeUnitInsertionException;
import ghidra.program.util.ExternalSymbolResolver;
import ghidra.util.*;
import ghidra.util.datastruct.*;
import ghidra.util.exception.*;
@@ -458,7 +457,8 @@ class ElfProgramBuilder extends MemorySectionResolver implements ElfLoadHelper {
String[] neededLibs = elf.getDynamicLibraryNames();
for (String neededLib : neededLibs) {
monitor.checkCancelled();
props.setString(ExternalSymbolResolver.getRequiredLibraryProperty(libraryIndex++),
props.setString(
AbstractLibrarySupportLoader.getRequiredLibraryProperty(libraryIndex++),
neededLib);
}
}

3
Ghidra/Features/Base/src/main/java/ghidra/app/util/opinion/MachoProgramBuilder.java
View File

@@ -57,7 +57,6 @@ import ghidra.program.model.reloc.RelocationResult;
import ghidra.program.model.reloc.RelocationTable;
import ghidra.program.model.symbol.*;
import ghidra.program.model.util.CodeUnitInsertionException;
import ghidra.program.util.ExternalSymbolResolver;
import ghidra.util.*;
import ghidra.util.exception.*;
import ghidra.util.task.TaskMonitor;
@@ -1298,7 +1297,7 @@ public class MachoProgramBuilder {
libraryPaths.add(libraryPath);
addLibrary(libraryPath);
props.setString(
ExternalSymbolResolver.getRequiredLibraryProperty(libraryIndex++),
AbstractLibrarySupportLoader.getRequiredLibraryProperty(libraryIndex++),
libraryPath);
}
}

169
Ghidra/Features/Base/src/main/java/ghidra/program/util/ExternalSymbolResolver.java
View File

@@ -23,75 +23,18 @@ import java.util.function.Consumer;
import db.Transaction;
import ghidra.app.util.opinion.Loaded;
import ghidra.framework.model.*;
import ghidra.framework.options.Options;
import ghidra.program.model.listing.*;
import ghidra.program.model.symbol.*;
import ghidra.util.Msg;
import ghidra.util.StringUtilities;
import ghidra.util.exception.*;
import ghidra.util.task.TaskMonitor;
/**
* Moves dangling external function symbols found in the {@link Library#UNKNOWN EXTERNAL/UNKNOWN}
* namespace into the namespace of the external library that publishes a matching symbol.
* <p>
* This uses an ordered list of external library names that was attached to the program during
* import by the Elf or Macho loader (see {@link #REQUIRED_LIBRARY_PROPERTY_PREFIX}).
*
*/
public class ExternalSymbolResolver implements Closeable {
private final static String REQUIRED_LIBRARY_PROPERTY_PREFIX = "Required Library [";
/**
* Gets a program property name to represent the ordered required library of the given index
*
* @param libraryIndex The index of the required library
* @return A program property name to represent the ordered required library of the given index
*/
public static String getRequiredLibraryProperty(int libraryIndex) {
return String.format("%s %s]", REQUIRED_LIBRARY_PROPERTY_PREFIX,
StringUtilities.pad("" + libraryIndex, ' ', 4));
}
/**
* Returns an ordered list of library names, as specified by the logic/rules of the original
* operating system's loader (eg. Elf / MachO dynamic library loading / symbol resolving
* rules)
*
* @param program The {@link Program}
* @return list of library names, in original order
*/
public static List<String> getOrderedRequiredLibraryNames(Program program) {
TreeMap<Integer, String> orderLibraryMap = new TreeMap<>();
Options options = program.getOptions(Program.PROGRAM_INFO);
for (String optionName : options.getOptionNames()) {
// Legacy programs may have the old "ELF Required Library [" program property, so
// we should not assume that the option name starts exactly with
// REQUIRED_LIBRARY_PROPERTY_PREFIX. We must deal with a potential substring at the
// start of the option name.
int prefixIndex = optionName.indexOf(REQUIRED_LIBRARY_PROPERTY_PREFIX);
if (prefixIndex == -1 || !optionName.endsWith("]")) {
continue;
}
String libName = options.getString(optionName, null);
if (libName == null) {
continue;
}
String indexStr = optionName
.substring(prefixIndex + REQUIRED_LIBRARY_PROPERTY_PREFIX.length(),
optionName.length() - 1)
.trim();
try {
orderLibraryMap.put(Integer.parseInt(indexStr), libName.trim());
}
catch (NumberFormatException e) {
Msg.error(ExternalSymbolResolver.class,
"Program contains invalid property: " + optionName);
}
}
return new ArrayList<>(orderLibraryMap.values());
}
private final ProjectData projectData;
private final TaskMonitor monitor;
@@ -236,33 +179,6 @@ public class ExternalSymbolResolver implements Closeable {
* Represents a program that needs its external symbols to be fixed.
*/
private class ProgramSymbolResolver {
record ExtLibInfo(String name, Library lib, String programPath, Program program,
List<String> resolvedSymbols, Throwable problem) {
String getProblemMessage() {
if (problem instanceof VersionException ve) {
return getVersionError(ve);
}
return problem != null ? problem.getMessage() : "";
}
String getLibPath() {
return programPath != null ? programPath : "missing";
}
String getVersionError(VersionException ve) {
String versionType = switch (ve.getVersionIndicator()) {
case VersionException.NEWER_VERSION -> " newer";
case VersionException.OLDER_VERSION -> "n older";
default -> "n unknown";
};
String upgradeMsg = ve.isUpgradable() ? " (upgrade is possible)" : "";
return "skipped: file was created with a%s version of Ghidra%s"
.formatted(versionType, upgradeMsg);
}
}
Program program;
String programPath;
@@ -296,16 +212,19 @@ public class ExternalSymbolResolver implements Closeable {
logger.accept("\t%d external symbols resolved, %d remain unresolved"
.formatted(getResolvedSymbolCount(), unresolvedExternalFunctionIds.size()));
for (ExtLibInfo extLib : extLibs) {
String libPath = extLib.getAssociatedProgramPath();
String loggedLibPath = libPath != null ? libPath : "missing";
if (extLib.problem != null) {
logger.accept("\t[%s] -> %s, %s".formatted(extLib.name, extLib.getLibPath(),
logger.accept("\t[%s] -> %s, %s".formatted(extLib.getName(), loggedLibPath,
extLib.getProblemMessage()));
}
else if (extLib.programPath != null) {
logger.accept("\t[%s] -> %s, %d new symbols resolved".formatted(extLib.name,
extLib.getLibPath(), extLib.resolvedSymbols.size()));
else if (libPath != null) {
logger.accept(
"\t[%s] -> %s, %d new symbols resolved".formatted(extLib.getName(),
loggedLibPath, extLib.resolvedSymbols.size()));
}
else {
logger.accept("\t[%s] -> %s".formatted(extLib.name, extLib.getLibPath()));
logger.accept("\t[%s] -> %s".formatted(extLib.getName(), loggedLibPath));
}
if (!shortSummary) {
for (String symbolName : extLib.resolvedSymbols) {
@@ -328,8 +247,8 @@ public class ExternalSymbolResolver implements Closeable {
private boolean hasSomeLibrariesConfigured() {
for (ExtLibInfo extLib : extLibs) {
if (extLib.program != null || extLib.problem != null ||
extLib.programPath != null) {
if (extLib.problem != null ||
extLib.getAssociatedProgramPath() != null) {
return true;
}
}
@@ -367,19 +286,17 @@ public class ExternalSymbolResolver implements Closeable {
private List<ExtLibInfo> getLibsToSearch() throws CancelledException {
List<ExtLibInfo> result = new ArrayList<>();
ExternalManager externalManager = program.getExternalManager();
for (String libName : getOrderedRequiredLibraryNames(program)) {
Library lib = externalManager.getExternalLibrary(libName);
String libPath = lib != null ? lib.getAssociatedProgramPath() : null;
// External manager supplies external Libraries in appropriate search order
for (Library lib : externalManager.getLibraries()) {
String libPath = lib.getAssociatedProgramPath();
Program libProg = libPath != null ? getLibraryProgram(libPath) : null;
Throwable problem =
libProg == null && libPath != null ? problemLibraries.get(libPath) : null;
result.add(
new ExtLibInfo(libName, lib, libPath, libProg, new ArrayList<>(), problem));
result.add(new ExtLibInfo(lib, problem));
}
return result;
}
/**
* Moves unresolved functions from the EXTERNAL/UNKNOWN namespace to the namespace of the
* external library if the extLib publishes a symbol with a matching name.
@@ -388,10 +305,6 @@ public class ExternalSymbolResolver implements Closeable {
* @throws CancelledException if cancelled
*/
private void resolveSymbolsToLibrary(ExtLibInfo extLib) throws CancelledException {
if (extLib.program == null) {
// can't do anything if the external library doesn't have a valid program associated
return;
}
ExternalManager externalManager = program.getExternalManager();
SymbolTable symbolTable = program.getSymbolTable();
@@ -409,7 +322,7 @@ public class ExternalSymbolResolver implements Closeable {
ExternalLocation extLoc = externalManager.getExternalLocation(s);
String extLocName =
Objects.requireNonNullElse(extLoc.getOriginalImportedName(), extLoc.getLabel());
if (isExportedSymbol(extLib.program, extLocName)) {
if (isExportedSymbol(program, extLocName)) {
try {
s.setNamespace(extLib.lib);
idIterator.remove();
@@ -443,6 +356,56 @@ public class ExternalSymbolResolver implements Closeable {
}
return symbolIds;
}
private class ExtLibInfo {
final Library lib;
final List<String> resolvedSymbols = new ArrayList<>();
final Throwable problem;
/**
* Define external Library dependency associated with {@link ProgramSymbolResolver}
* instance.
* @param lib external library dependency
* @param problem exception which occured while accessing Library
*/
ExtLibInfo(Library lib, Throwable problem) {
if (program != lib.getSymbol().getProgram()) {
throw new AssertionError("Program mismatch");
}
this.lib = lib;
this.problem = problem;
}
String getName() {
return lib.getName();
}
String getProblemMessage() {
if (problem instanceof VersionException ve) {
return getVersionError(ve);
}
return problem != null ? problem.getMessage() : "";
}
String getVersionError(VersionException ve) {
String versionType = switch (ve.getVersionIndicator()) {
case VersionException.NEWER_VERSION -> " newer";
case VersionException.OLDER_VERSION -> "n older";
default -> "n unknown";
};
String upgradeMsg = ve.isUpgradable() ? " (upgrade is possible)" : "";
return "skipped: file was created with a%s version of Ghidra%s"
.formatted(versionType, upgradeMsg);
}
String getAssociatedProgramPath() {
return lib.getAssociatedProgramPath();
}
}
}
/**

15
Ghidra/Features/Base/src/main/java/ghidra/program/util/SymbolMerge.java
View File

@@ -230,11 +230,20 @@ class SymbolMerge {
}
else if (type == SymbolType.LIBRARY) {
ExternalManager fromExtMgr = fromProgram.getExternalManager();
String path = fromExtMgr.getExternalLibraryPath(name);
Library fromLib = fromExtMgr.getExternalLibrary(name);
String path = fromLib != null ? fromLib.getAssociatedProgramPath() : null;
int ordinal = fromLib != null ? fromExtMgr.getLibraryOrdinal(name) : -1;
ExternalManagerDB extMgr = (ExternalManagerDB) toProgram.getExternalManager();
extMgr.setExternalPath(name, path, source == SourceType.USER_DEFINED);
symbol = toSymbolTable.getLibrarySymbol(name);
Library newLib = extMgr.addExternalLibraryName(name, source);
symbol = newLib.getSymbol();
if (ordinal >= 0) {
extMgr.setLibraryOrdinal(name, ordinal);
}
if (path != null) {
extMgr.setExternalPath(name, path, source == SourceType.USER_DEFINED);
}
}
else if (type == SymbolType.FUNCTION) {
FunctionManager fromFunctionMgr = fromProgram.getFunctionManager();

9
Ghidra/Features/Base/src/test.slow/java/ghidra/program/database/external/ExternalManagerDBTest.java vendored
View File

@@ -4,9 +4,9 @@
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
*
* http://www.apache.org/licenses/LICENSE-2.0
*
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -25,7 +25,6 @@ import ghidra.program.database.ProgramBuilder;
import ghidra.program.database.ProgramDB;
import ghidra.program.model.address.Address;
import ghidra.program.model.address.AddressSpace;
import ghidra.program.model.listing.CircularDependencyException;
import ghidra.program.model.listing.Library;
import ghidra.program.model.symbol.*;
import ghidra.test.AbstractGhidraHeadedIntegrationTest;
@@ -47,7 +46,7 @@ public class ExternalManagerDBTest extends AbstractGhidraHeadedIntegrationTest {
public void setUp() throws Exception {
program = createDefaultProgram(testName.getMethodName(), ProgramBuilder._TOY, this);
space = program.getAddressFactory().getDefaultAddressSpace();
extMgr = (ExternalManagerDB) program.getExternalManager();
extMgr = program.getExternalManager();
transactionID = program.startTransaction("Test");
}
@@ -336,7 +335,7 @@ public class ExternalManagerDBTest extends AbstractGhidraHeadedIntegrationTest {
@Test
public void testOriginalImportName()
throws InvalidInputException, DuplicateNameException, CircularDependencyException {
throws InvalidInputException, DuplicateNameException {
ExternalLocation extLoc =
extMgr.addExtLocation("ext1", "foo", addr(1000), SourceType.IMPORTED);

38
Ghidra/Features/Sarif/src/main/java/sarif/managers/ExternalLibSarifMgr.java
View File

@@ -4,9 +4,9 @@
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
*
* http://www.apache.org/licenses/LICENSE-2.0
*
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -16,28 +16,15 @@
package sarif.managers;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.*;
import com.google.gson.JsonArray;
import ghidra.app.util.importer.MessageLog;
import ghidra.program.model.address.Address;
import ghidra.program.model.address.AddressFormatException;
import ghidra.program.model.address.AddressOverflowException;
import ghidra.program.model.listing.GhidraClass;
import ghidra.program.model.listing.Library;
import ghidra.program.model.listing.Program;
import ghidra.program.model.symbol.ExternalLocation;
import ghidra.program.model.symbol.ExternalManager;
import ghidra.program.model.symbol.Namespace;
import ghidra.program.model.symbol.SourceType;
import ghidra.util.exception.CancelledException;
import ghidra.util.exception.DuplicateNameException;
import ghidra.util.exception.InvalidInputException;
import ghidra.program.model.address.*;
import ghidra.program.model.listing.*;
import ghidra.program.model.symbol.*;
import ghidra.util.exception.*;
import ghidra.util.task.TaskLauncher;
import ghidra.util.task.TaskMonitor;
import sarif.SarifProgramOptions;
@@ -127,14 +114,21 @@ public class ExternalLibSarifMgr extends SarifMgr {
return; // already has a value--don't override it
}
// NB: Can't use "DEFAULT" here or result.get("sourceType") which may be
// "DEFAULT"
// NB: Can't use "DEFAULT" here or result.get("sourceType") which may be "DEFAULT"
String source = (String) result.get("sourceType");
SourceType sourceType = getSourceType(source);
if (sourceType.equals(SourceType.DEFAULT)) {
sourceType = SourceType.IMPORTED;
}
library = extManager.addExternalLibraryName(progName, sourceType);
// NOTE: It is assumed that a full export/import will maintain the ordered sequence of Libraries
// NOTE: Skip setting library location within project which is unlikely to match-up properly
// String progPath = (String) result.get("location");
// if (progPath != null) {
// extManager.setExternalPath(progName, progPath, sourceType == SourceType.USER_DEFINED);
// }
}
private void processExternalLocation(Map<String, Object> result) throws IOException {

4
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/ProgramDB.java
View File

@@ -120,8 +120,9 @@ public class ProgramDB extends DomainObjectAdapterDB implements Program, ChangeM
* 15-Sep-2025 - version 31 Code Mananger dropped Composites property map use
* 19-Sep-2025 - version 32 Expanded number of SourceType values and record storage affecting
* SymbolDB, FunctionDB and RefListFlagsV0
* 14-Apr-2026 - version 33 Introduced Library symbol ordinal assignment.
*/
static final int DB_VERSION = 32;
static final int DB_VERSION = 33;
/**
* UPGRADE_REQUIRED_BFORE_VERSION should be changed to DB_VERSION anytime the
@@ -142,6 +143,7 @@ public class ProgramDB extends DomainObjectAdapterDB implements Program, ChangeM
public static final int COMPOUND_VARIABLE_STORAGE_ADDED_VERSION = 18;
public static final int AUTO_PARAMETERS_ADDED_VERSION = 19;
public static final int RELOCATION_STATUS_ADDED_VERSION = 26;
public static final int LIBRARY_ORDINAL_ASSIGNMENT_ADDED_VERSION = 33;
private static final String DATA_MAP_TABLE_NAME = "Program";

6
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/external/ExternalLocationDB.java vendored
View File

@@ -55,6 +55,12 @@ public class ExternalLocationDB implements ExternalLocation {
return library != null ? library.getName() : "<UNKNOWN>";
}
@Override
public String getExternalLibraryPath() {
Library library = getLibrary();
return library != null ? library.getAssociatedProgramPath() : null;
}
private Library getLibrary() {
Namespace parent = symbol.getParentNamespace();
while (parent != null && !(parent instanceof Library)) {

135
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/external/ExternalManagerDB.java vendored
View File

@@ -22,7 +22,6 @@ import org.apache.commons.lang3.StringUtils;
import db.*;
import ghidra.framework.data.OpenMode;
import ghidra.framework.store.FileSystem;
import ghidra.program.database.ManagerDB;
import ghidra.program.database.ProgramDB;
import ghidra.program.database.function.FunctionDB;
@@ -126,7 +125,7 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
String name = rec.getString(OldExtNameAdapter.EXT_NAME_COL);
try {
addExternalName(name, rec.getString(OldExtNameAdapter.EXT_PATHNAME_COL),
doAddExternalName(name, rec.getString(OldExtNameAdapter.EXT_PATHNAME_COL),
SourceType.USER_DEFINED);
nameMap.put(rec.getKey(), name);
}
@@ -200,11 +199,8 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
SourceType sourceType) throws InvalidInputException, DuplicateNameException {
SourceType locSourceType = checkExternalLabel(extLabel, extAddr, sourceType);
try (Closeable c = lock.write()) {
Library libraryScope = getLibraryScope(extLibraryName);
if (libraryScope == null) {
libraryScope = addExternalName(extLibraryName, null, sourceType);
}
return addExtLocation(libraryScope, extLabel, extAddr, false, locSourceType, true);
Library library = addExternalLibraryName(extLibraryName, sourceType);
return addExtLocation(library, extLabel, extAddr, false, locSourceType, true);
}
}
@@ -231,12 +227,9 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
SourceType sourceType) throws InvalidInputException, DuplicateNameException {
SourceType locSourceType = checkExternalLabel(extLabel, extAddr, sourceType);
try (Closeable c = lock.write()) {
Library libraryScope = getLibraryScope(extLibraryName);
if (libraryScope == null) {
libraryScope = addExternalName(extLibraryName, null,
Library library = addExternalLibraryName(extLibraryName,
sourceType != SourceType.DEFAULT ? sourceType : SourceType.ANALYSIS);
}
return addExtLocation(libraryScope, extLabel, extAddr, true, locSourceType, true);
return addExtLocation(library, extLabel, extAddr, true, locSourceType, true);
}
}
@@ -272,16 +265,11 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
Address extAddr, boolean isFunction, SourceType sourceType, boolean reuseExisting)
throws InvalidInputException {
if (extNamespace == null) {
extNamespace = getLibraryScope(Library.UNKNOWN);
if (extNamespace == null) {
try {
extNamespace = addExternalLibraryName(Library.UNKNOWN, SourceType.ANALYSIS);
}
catch (DuplicateNameException e) {
// TODO: really need to reserve the unknown namespace name
throw new InvalidInputException(
"Failed to establish " + Library.UNKNOWN + " library");
}
try {
extNamespace = addExternalLibraryName(Library.UNKNOWN, SourceType.ANALYSIS);
}
catch (InvalidInputException | DuplicateNameException e) {
throw new AssertionError(e); // reserved name should be OK
}
}
else if (!extNamespace.isExternal()) {
@@ -471,7 +459,7 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
@Override
public Set<ExternalLocation> getExternalLocations(String libraryName, String label) {
Library library = getLibraryScope(libraryName);
Library library = getExternalLibrary(libraryName);
if (library == null && !StringUtils.isBlank(libraryName)) {
return Set.of();
}
@@ -489,7 +477,7 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
@Override
public ExternalLocation getUniqueExternalLocation(String libraryName, String label) {
Library library = getLibraryScope(libraryName);
Library library = getExternalLibrary(libraryName);
if (library == null && !StringUtils.isBlank(libraryName)) {
return null;
}
@@ -598,12 +586,16 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
}
@Override
public void updateExternalLibraryName(String oldName, String newName, SourceType source)
public boolean updateExternalLibraryName(String oldName, String newName, SourceType source)
throws DuplicateNameException, InvalidInputException {
Symbol s = symbolMgr.getLibrarySymbol(oldName);
if (s != null) {
s.setName(newName, source);
try (Closeable c = lock.write()) {
Symbol s = symbolMgr.getLibrarySymbol(oldName);
if (s != null) {
s.setName(newName, source);
return true;
}
}
return false;
}
@Override
@@ -614,34 +606,37 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
if (librarySymbol != null) {
return (Library) librarySymbol.getObject();
}
return addExternalName(name, null, source);
return doAddExternalName(name, null, source);
}
}
private Library addExternalName(String name, String pathname, SourceType source)
private Library doAddExternalName(String name, String pathname, SourceType source)
throws DuplicateNameException, InvalidInputException {
SymbolDB s = symbolMgr.createLibrarySymbol(name, pathname, source);
return (Library) s.getObject();
}
private Library getLibraryScope(String name) {
LibrarySymbol s = symbolMgr.getLibrarySymbol(name);
return s == null ? null : s.getObject();
}
@Override
public boolean contains(String libraryName) {
return symbolMgr.getLibrarySymbol(libraryName) != null;
}
@Override
public List<Library> getLibraries() {
try (Closeable c = lock.read()) {
List<Library> orderedLibraries = new ArrayList<>();
for (LibrarySymbol libSym : symbolMgr.getLibrarySymbolList()) {
orderedLibraries.add(libSym.getObject());
}
return orderedLibraries;
}
}
@Override
public String[] getExternalLibraryNames() {
ArrayList<String> list = new ArrayList<>();
Symbol[] syms = symbolMgr.getSymbols(Address.NO_ADDRESS);
for (Symbol s : syms) {
if (s.getSymbolType() == SymbolType.LIBRARY) {
list.add(s.getName());
}
for (LibrarySymbol libSym : symbolMgr.getLibrarySymbolList()) {
list.add(libSym.getName());
}
String[] names = new String[list.size()];
list.toArray(names);
@@ -650,8 +645,10 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
@Override
public Library getExternalLibrary(String name) {
Symbol s = symbolMgr.getLibrarySymbol(name);
return s != null ? (Library) s.getObject() : null;
try (Closeable c = lock.read()) {
Symbol s = symbolMgr.getLibrarySymbol(name);
return s != null ? (Library) s.getObject() : null;
}
}
@Override
@@ -672,34 +669,38 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
return;
}
validateExternalPath(externalPath);
LibrarySymbol.validateExternalPath(externalPath);
try (Closeable c = lock.write()) {
LibrarySymbol s = symbolMgr.getLibrarySymbol(externalName);
if (s == null) {
try {
addExternalName(externalName, externalPath,
userDefined ? SourceType.USER_DEFINED : SourceType.IMPORTED);
}
catch (DuplicateNameException e) {
throw new AssertException(e);
}
}
else {
s.setExternalLibraryPath(externalPath);
}
Library library = addExternalLibraryName(externalName,
userDefined ? SourceType.USER_DEFINED : SourceType.IMPORTED);
LibrarySymbol libSym = (LibrarySymbol) library.getSymbol();
libSym.setExternalLibraryPath(externalPath);
}
catch (DuplicateNameException e) {
// ignore - new externalName conflicts with another namespace
}
}
private void validateExternalPath(String path) throws InvalidInputException {
if (path == null) {
return; // null is an allowed value (used to clear)
}
@Override
public int getLibraryOrdinal(String libraryName) {
LibrarySymbol libSym = symbolMgr.getLibrarySymbol(libraryName);
return libSym != null ? libSym.getOrdinal() : -1;
}
int len = path.length();
if (len == 0 || path.charAt(0) != FileSystem.SEPARATOR_CHAR) {
throw new InvalidInputException(
"Absolute path must begin with '" + FileSystem.SEPARATOR_CHAR + "'");
@Override
public int setLibraryOrdinal(String libraryName, int ordinal) {
if (Library.UNKNOWN.equals(libraryName)) {
Msg.warn(this, "Ignoring external library ordinal assignment for " + libraryName);
return -1;
}
try (Closeable c = lock.write()) {
LibrarySymbol libSym = symbolMgr.getLibrarySymbol(libraryName);
if (libSym == null) {
return -1;
}
libSym.setOrdinal(ordinal);
return libSym.getOrdinal();
}
}
@@ -740,9 +741,9 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
@Override
public ExternalLocationIterator getExternalLocations(String externalName) {
Library scope = getLibraryScope(externalName);
if (scope != null) {
return new ExternalLocationDBIterator(symbolMgr.getSymbols(scope));
Library library = getExternalLibrary(externalName);
if (library != null) {
return new ExternalLocationDBIterator(symbolMgr.getSymbols(library));
}
return new ExternalLocationDBIterator();
}

5
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/LibraryDB.java
View File

@@ -99,6 +99,11 @@ class LibraryDB implements Library {
return symbol.getExternalLibraryPath();
}
@Override
public void setAssociatedProgramPath(String programPath) throws InvalidInputException {
symbol.setExternalLibraryPath(programPath);
}
@Override
public boolean isExternal() {
return true;

207
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/LibrarySymbol.java
View File

@@ -15,7 +15,11 @@
*/
package ghidra.program.database.symbol;
import java.util.*;
import db.DBRecord;
import db.Field;
import ghidra.framework.store.FileSystem;
import ghidra.program.model.address.Address;
import ghidra.program.model.listing.CircularDependencyException;
import ghidra.program.model.listing.Library;
@@ -32,7 +36,7 @@ import ghidra.util.exception.InvalidInputException;
* Symbol data usage:
* String stringData - associated program project file path
*/
public class LibrarySymbol extends SymbolDB {
public class LibrarySymbol extends SymbolDB implements Comparable<LibrarySymbol> {
private LibraryDB library;
@@ -46,35 +50,69 @@ public class LibrarySymbol extends SymbolDB {
}
@Override
public void setName(String newName, SourceType source)
throws DuplicateNameException, InvalidInputException {
public void setNameAndNamespace(String newName, Namespace newNamespace, SourceType source)
throws DuplicateNameException, InvalidInputException, CircularDependencyException {
String oldName = getName();
if (Library.UNKNOWN.equals(oldName)) {
Msg.warn(this, "Unable to change name of " + Library.UNKNOWN + " Library");
return;
}
if (newNamespace.getID() != Namespace.GLOBAL_NAMESPACE_ID) {
throw new InvalidInputException("Namespace \"" + newNamespace.getName(true) +
"\" is not valid for library " + getName());
}
super.setName(newName, source);
try (Closeable c = lock.write()) {
if (!oldName.equals(getName())) {
symbolMgr.getProgram()
.setObjChanged(ProgramEvent.EXTERNAL_NAME_CHANGED, (Address) null, null,
oldName, newName);
super.setNameAndNamespace(newName, newNamespace, source);
if (!oldName.equals(getName())) {
symbolMgr.getProgram()
.setObjChanged(ProgramEvent.EXTERNAL_NAME_CHANGED, (Address) null, this,
oldName, newName);
}
if (Library.UNKNOWN.equals(newName)) {
symbolMgr.adjustLibraryOrdinals(this, 0);
setExternalLibraryPath(null); // clear file path for UNKNOWN lib
}
}
}
@Override
public void setNameAndNamespace(String newName, Namespace newNamespace, SourceType source)
throws DuplicateNameException, InvalidInputException, CircularDependencyException {
String oldName = getName();
public boolean delete() {
try (Closeable c = lock.write()) {
super.setNameAndNamespace(newName, newNamespace, source);
// Pre-fetch library symbol list to facilitate ordinal reassignments after removal
int ordinal = getOrdinal();
List<LibrarySymbol> libSymList = new ArrayList<>(symbolMgr.getLibrarySymbolList());
if (!oldName.equals(getName())) {
symbolMgr.getProgram()
.setObjChanged(ProgramEvent.EXTERNAL_NAME_CHANGED, (Address) null, null,
oldName, newName);
if (super.delete()) {
// Perform ordinal reassignments for remaining library symbols if needed.
// It is expected that all ordinals are accounted for in cached library list
// but to be safe we will perform brute-force search if mismatch and ignore
// if not found in list;
LibrarySymbol s = libSymList.get(ordinal);
if (s != this) {
String libName = record.getString(SymbolDatabaseAdapter.SYMBOL_NAME_COL);
Msg.error(this,
"Library symbol list did not contain removed symbol: " + libName);
for (ordinal = 0; ordinal < libSymList.size(); ordinal++) {
s = libSymList.get(ordinal);
if (s == this) {
break;
}
}
}
if (s == this) {
libSymList.remove(ordinal);
symbolMgr.assignLibraryOrdinals(libSymList, false);
}
return true;
}
}
return false;
}
@Override
@@ -111,6 +149,76 @@ public class LibrarySymbol extends SymbolDB {
SymbolType.LIBRARY.isValidParent(symbolMgr.getProgram(), parent, address, isExternal());
}
/**
* {@return Library's ordinal placement within ordered library list.}
*/
public int getOrdinal() {
// NOTE: This method must not be used by compareTo since it may invoke it
validate(lock);
int ordinal = doGetOrdinalFromRecord();
if (ordinal < 0) {
// NOTE: this method invocation relies on the use of the compareTo method
ordinal = symbolMgr.computeLibraryOrdinal(this);
}
return ordinal;
}
/**
* Get this Library's ordinal as stored in the database. A value of -1 is returned if one has
* not yet been established in which case the symbol ID should be used for sort while ensuring
* {@link Library#UNKNOWN} always sorts as first.
*
* @return Library symbol stored ordinal or -1 if not yet stored.
*/
int doGetOrdinalFromRecord() {
Field fieldValue = record.getFieldValue(SymbolDatabaseAdapter.SYMBOL_LIB_ORDINAL_COL);
return fieldValue != null ? fieldValue.getIntValue() : -1;
}
/**
* Set the Library ordinal.
* Other Library ordinals will be adjusted if displaced by ordinal change.
* No change is made if this Library symbol corresponds to {@link Library#UNKNOWN} Library.
*
* @param ordinal positive greater or equal to 0.
* @throws IllegalArgumentException if a negative ordinal is specified
*/
public void setOrdinal(int ordinal) {
if (ordinal < 0) {
throw new IllegalArgumentException("Non-negative ordinal is required");
}
if (Library.UNKNOWN.equals(getName())) {
return; // ordinal change ignored for UNKNOWN Library
}
try (Closeable c = lock.write()) {
checkDeleted();
if (ordinal == 0) {
// Cannot displace UNKNOWN Library which may reside at ordinal 0
LibrarySymbol displacedLibSym = symbolMgr.getLibrarySymbolByOrdinal(0);
if (displacedLibSym != null && Library.UNKNOWN.equals(displacedLibSym.getName())) {
return;
}
}
symbolMgr.adjustLibraryOrdinals(this, ordinal);
}
}
void doSetOrdinal(int newOrdinal, boolean notify) {
if (newOrdinal < 0) {
throw new IllegalArgumentException("Unsupported ordinal assignment: " + newOrdinal);
}
int oldOrdinal = doGetOrdinalFromRecord();
if (oldOrdinal == newOrdinal) {
return;
}
record.setIntValue(SymbolDatabaseAdapter.SYMBOL_LIB_ORDINAL_COL, newOrdinal);
updateRecord();
if (notify) {
symbolMgr.symbolDataChanged(this);
}
}
/**
* {@return the library program path within the project (may be null)}
*/
@@ -121,22 +229,73 @@ public class LibrarySymbol extends SymbolDB {
/**
* Set the library program path within the project.
* The {@link Library#UNKNOWN} Library path may only be cleared.
*
* @param libraryPath library program path or null to clear
* @throws InvalidInputException if an invalid project file path is specified
*/
public void setExternalLibraryPath(String libraryPath) {
String oldPath = getExternalLibraryPath();
public void setExternalLibraryPath(String libraryPath) throws InvalidInputException {
try (Closeable c = lock.write()) {
checkDeleted();
setRecordFields(record, libraryPath);
if (Library.UNKNOWN.equals(getName())) {
libraryPath = null;
}
validateExternalPath(libraryPath);
String oldPath = record.getString(SymbolDatabaseAdapter.SYMBOL_LIBPATH_COL);
if (Objects.equals(oldPath, libraryPath)) {
return;
}
record.setString(SymbolDatabaseAdapter.SYMBOL_LIBPATH_COL, libraryPath);
updateRecord();
}
symbolMgr.getProgram()
.setObjChanged(ProgramEvent.EXTERNAL_PATH_CHANGED, getName(), oldPath, libraryPath);
symbolMgr.symbolDataChanged(this);
}
static void setRecordFields(DBRecord record, String libraryPath) {
/**
* Perform path validation for an external library path within the project
* @param path external library path within the project (null is allowed for clearing path)
* @throws InvalidInputException if path is invalid
*/
public static void validateExternalPath(String path) throws InvalidInputException {
if (path == null) {
return; // null is an allowed value (used to clear)
}
int len = path.length();
if (len == 0 || path.charAt(0) != FileSystem.SEPARATOR_CHAR) {
throw new InvalidInputException(
"Absolute path must begin with '" + FileSystem.SEPARATOR_CHAR + "'");
}
}
/**
* Set library symbol record fields during symbol creation
* @param record new symbol record
* @param ordinal library symbol ordinal
* @param libraryPath library path or null
*/
static void setRecordFields(DBRecord record, int ordinal, String libraryPath) {
// NOTE: method use must be limited since ordinal re-assignments of affected Libraries
// is not handled here.
record.setIntValue(SymbolDatabaseAdapter.SYMBOL_LIB_ORDINAL_COL, ordinal);
record.setString(SymbolDatabaseAdapter.SYMBOL_LIBPATH_COL, libraryPath);
}
@Override
public int compareTo(LibrarySymbol o) {
validate(lock);
o.validate(lock);
// NOTE: this method is not intended to be used between symbols from different programs
// where one may have stored ordinals and the other may not in which case this comparison
// would be invalid. For a single program it is required that all library symbols either
// have an assigned ordinal or do not in which cases symbol ID comparison is used
int c = Long.compare(doGetOrdinalFromRecord(), o.doGetOrdinalFromRecord());
if (c == 0) {
// Handles case where all library symbols report a -1 ordinal (not yet assigned)
// UNKNOWN Library placement is arbitrary in this case.
c = Long.compare(getID(), o.getID());
}
return c;
}
}

21
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/SymbolDatabaseAdapter.java
View File

@@ -16,7 +16,8 @@
package ghidra.program.database.symbol;
import java.io.IOException;
import java.util.*;
import java.util.Objects;
import java.util.Set;
import org.apache.commons.lang3.StringUtils;
@@ -52,8 +53,9 @@ abstract class SymbolDatabaseAdapter {
static final int SYMBOL_EXTERNAL_PROG_ADDR_COL = 10;
static final int SYMBOL_COMMENT_COL = 11;
static final int SYMBOL_LIBPATH_COL = 12;
static final int SYMBOL_LIB_ORDINAL_COL = 13;
static final Schema SYMBOL_SCHEMA = SymbolDatabaseAdapterV4.V4_SYMBOL_SCHEMA;
static final Schema SYMBOL_SCHEMA = SymbolDatabaseAdapterV5.V5_SYMBOL_SCHEMA;
// Bits 0, 1 and 3 are used for the source of the symbol.
// NOTE: On the next V5 adapter revision the source type bits should be made contiguous
@@ -96,11 +98,11 @@ abstract class SymbolDatabaseAdapter {
throws VersionException, CancelledException, IOException {
if (openMode == OpenMode.CREATE) {
return new SymbolDatabaseAdapterV4(dbHandle, addrMap, true);
return new SymbolDatabaseAdapterV5(dbHandle, addrMap, true);
}
try {
SymbolDatabaseAdapter adapter = new SymbolDatabaseAdapterV4(dbHandle, addrMap, false);
SymbolDatabaseAdapter adapter = new SymbolDatabaseAdapterV5(dbHandle, addrMap, false);
return adapter;
}
catch (VersionException e) {
@@ -122,6 +124,13 @@ abstract class SymbolDatabaseAdapter {
private static SymbolDatabaseAdapter findReadOnlyAdapter(DBHandle handle, AddressMap addrMap)
throws VersionException {
try {
return new SymbolDatabaseAdapterV4(handle, addrMap.getOldAddressMap());
}
catch (VersionException e) {
// failed try older version
}
try {
return new SymbolDatabaseAdapterV3(handle, addrMap.getOldAddressMap());
}
@@ -168,7 +177,7 @@ abstract class SymbolDatabaseAdapter {
dbHandle.deleteTable(SYMBOL_TABLE_NAME);
SymbolDatabaseAdapter newAdapter = new SymbolDatabaseAdapterV4(dbHandle, addrMap, true);
SymbolDatabaseAdapter newAdapter = new SymbolDatabaseAdapterV5(dbHandle, addrMap, true);
copyTempToNewAdapter(tmpAdapter, newAdapter, monitor);
return newAdapter;
@@ -194,7 +203,7 @@ abstract class SymbolDatabaseAdapter {
((SymbolDatabaseAdapterV0) oldAdapter).extractLocalSymbols(tmpHandle, monitor);
}
SymbolDatabaseAdapterV4 tmpAdapter = new SymbolDatabaseAdapterV4(tmpHandle, addrMap, true);
SymbolDatabaseAdapterV5 tmpAdapter = new SymbolDatabaseAdapterV5(tmpHandle, addrMap, true);
RecordIterator iter = oldAdapter.getSymbols();
while (iter.hasNext()) {
monitor.checkCancelled();

22
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/SymbolDatabaseAdapterV2.java
View File

@@ -97,7 +97,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) {
return Field.EMPTY_ARRAY;
}
return symbolTable.findRecords(new LongField(key), SYMBOL_ADDR_COL);
return symbolTable.findRecords(new LongField(key), V2_SYMBOL_ADDR_COL);
}
@Override
@@ -113,7 +113,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
@Override
RecordIterator getSymbolsByAddress(boolean forward) throws IOException {
KeyToRecordIterator it = new KeyToRecordIterator(symbolTable,
new AddressIndexPrimaryKeyIterator(symbolTable, SYMBOL_ADDR_COL, addrMap, forward));
new AddressIndexPrimaryKeyIterator(symbolTable, V2_SYMBOL_ADDR_COL, addrMap, forward));
return new V2ConvertedRecordIterator(it);
}
@@ -121,7 +121,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
RecordIterator getSymbolsByAddress(Address startAddr, boolean forward) throws IOException {
KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, startAddr, forward));
V2_SYMBOL_ADDR_COL, addrMap, startAddr, forward));
return new V2ConvertedRecordIterator(it);
}
@@ -139,7 +139,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
RecordIterator getSymbols(Address start, Address end, boolean forward) throws IOException {
KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, start, end, forward));
V2_SYMBOL_ADDR_COL, addrMap, start, end, forward));
return new V2ConvertedRecordIterator(it);
}
@@ -147,7 +147,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
RecordIterator getSymbols(AddressSetView set, boolean forward) throws IOException {
KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, set, forward));
V2_SYMBOL_ADDR_COL, addrMap, set, forward));
return new V2ConvertedRecordIterator(it);
}
@@ -155,7 +155,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
RecordIterator getPrimarySymbols(AddressSetView set, boolean forward) throws IOException {
KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, set, forward));
V2_SYMBOL_ADDR_COL, addrMap, set, forward));
return getPrimaryFilterRecordIterator(new V2ConvertedRecordIterator(it));
}
@@ -222,21 +222,21 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
@Override
RecordIterator getSymbolsByNamespace(long id) throws IOException {
LongField field = new LongField(id);
RecordIterator it = symbolTable.indexIterator(SYMBOL_PARENT_ID_COL, field, field, true);
RecordIterator it = symbolTable.indexIterator(V2_SYMBOL_PARENT_ID_COL, field, field, true);
return new V2ConvertedRecordIterator(it);
}
@Override
RecordIterator getSymbolsByName(String name) throws IOException {
StringField field = new StringField(name);
RecordIterator it = symbolTable.indexIterator(SYMBOL_NAME_COL, field, field, true);
RecordIterator it = symbolTable.indexIterator(V2_SYMBOL_NAME_COL, field, field, true);
return new V2ConvertedRecordIterator(it);
}
@Override
RecordIterator scanSymbolsByName(String startName) throws IOException {
StringField field = new StringField(startName);
RecordIterator it = symbolTable.indexIterator(SYMBOL_NAME_COL, field, null, true);
RecordIterator it = symbolTable.indexIterator(V2_SYMBOL_NAME_COL, field, null, true);
return new V2ConvertedRecordIterator(it);
}
@@ -262,7 +262,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
Address getMaxSymbolAddress(AddressSpace space) throws IOException {
if (space.isMemorySpace()) {
AddressIndexKeyIterator addressKeyIterator = new AddressIndexKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, space.getMinAddress(), space.getMaxAddress(), false);
V2_SYMBOL_ADDR_COL, addrMap, space.getMinAddress(), space.getMaxAddress(), false);
if (addressKeyIterator.hasNext()) {
return addrMap.decodeAddress(addressKeyIterator.next());
}
@@ -270,7 +270,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
else {
LongField max = new LongField(addrMap.getKey(space.getMaxAddress(), false));
DBFieldIterator iterator =
symbolTable.indexFieldIterator(null, max, false, SYMBOL_ADDR_COL);
symbolTable.indexFieldIterator(null, max, false, V2_SYMBOL_ADDR_COL);
if (iterator.hasPrevious()) {
LongField val = (LongField) iterator.previous();
Address addr = addrMap.decodeAddress(val.getLongValue());

34
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/SymbolDatabaseAdapterV3.java
View File

@@ -110,7 +110,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) {
return false;
}
return symbolTable.hasRecord(new LongField(key), SYMBOL_ADDR_COL);
return symbolTable.hasRecord(new LongField(key), V3_SYMBOL_ADDR_COL);
}
@Override
@@ -119,7 +119,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) {
return Field.EMPTY_ARRAY;
}
return symbolTable.findRecords(new LongField(key), SYMBOL_ADDR_COL);
return symbolTable.findRecords(new LongField(key), V3_SYMBOL_ADDR_COL);
}
@Override
@@ -135,7 +135,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
@Override
RecordIterator getSymbolsByAddress(boolean forward) throws IOException {
KeyToRecordIterator it = new KeyToRecordIterator(symbolTable,
new AddressIndexPrimaryKeyIterator(symbolTable, SYMBOL_ADDR_COL, addrMap, forward));
new AddressIndexPrimaryKeyIterator(symbolTable, V3_SYMBOL_ADDR_COL, addrMap, forward));
return new V3ConvertedRecordIterator(it);
}
@@ -143,7 +143,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
RecordIterator getSymbolsByAddress(Address startAddr, boolean forward) throws IOException {
KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, startAddr, forward));
V3_SYMBOL_ADDR_COL, addrMap, startAddr, forward));
return new V3ConvertedRecordIterator(it);
}
@@ -161,7 +161,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
RecordIterator getSymbols(Address start, Address end, boolean forward) throws IOException {
KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, start, end, forward));
V3_SYMBOL_ADDR_COL, addrMap, start, end, forward));
return new V3ConvertedRecordIterator(it);
}
@@ -169,7 +169,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
RecordIterator getSymbols(AddressSetView set, boolean forward) throws IOException {
KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, set, forward));
V3_SYMBOL_ADDR_COL, addrMap, set, forward));
return new V3ConvertedRecordIterator(it);
}
@@ -178,14 +178,14 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
throws IOException {
KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_PRIMARY_COL, addrMap, set, forward));
V3_SYMBOL_PRIMARY_COL, addrMap, set, forward));
return new V3ConvertedRecordIterator(it);
}
@Override
protected DBRecord getPrimarySymbol(Address address) throws IOException {
AddressIndexPrimaryKeyIterator it = new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_PRIMARY_COL, addrMap, address, address, true);
V3_SYMBOL_PRIMARY_COL, addrMap, address, address, true);
if (it.hasNext()) {
return convertV3Record(symbolTable.getRecord(it.next()));
}
@@ -257,21 +257,21 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
@Override
RecordIterator getSymbolsByNamespace(long id) throws IOException {
LongField field = new LongField(id);
RecordIterator it = symbolTable.indexIterator(SYMBOL_PARENT_ID_COL, field, field, true);
RecordIterator it = symbolTable.indexIterator(V3_SYMBOL_PARENT_ID_COL, field, field, true);
return new V3ConvertedRecordIterator(it);
}
@Override
RecordIterator getSymbolsByName(String name) throws IOException {
StringField field = new StringField(name);
RecordIterator it = symbolTable.indexIterator(SYMBOL_NAME_COL, field, field, true);
RecordIterator it = symbolTable.indexIterator(V3_SYMBOL_NAME_COL, field, field, true);
return new V3ConvertedRecordIterator(it);
}
@Override
RecordIterator scanSymbolsByName(String startName) throws IOException {
StringField field = new StringField(startName);
RecordIterator it = symbolTable.indexIterator(SYMBOL_NAME_COL, field, null, true);
RecordIterator it = symbolTable.indexIterator(V3_SYMBOL_NAME_COL, field, null, true);
return new V3ConvertedRecordIterator(it);
}
@@ -286,7 +286,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
Field end = computeLocatorHash(name, id, MAX_ADDRESS_OFFSET);
RecordIterator it = symbolTable.indexIterator(SYMBOL_HASH_COL, start, end, true);
RecordIterator it = symbolTable.indexIterator(V3_SYMBOL_HASH_COL, start, end, true);
it = new V3ConvertedRecordIterator(it);
RecordIterator filtered = getNameAndNamespaceFilterIterator(name, id, it);
@@ -300,7 +300,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
if (search == null) {
return null;
}
RecordIterator it = symbolTable.indexIterator(SYMBOL_HASH_COL, search, search, true);
RecordIterator it = symbolTable.indexIterator(V3_SYMBOL_HASH_COL, search, search, true);
it = new V3ConvertedRecordIterator(it);
RecordIterator filtered =
@@ -315,7 +315,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
Address getMaxSymbolAddress(AddressSpace space) throws IOException {
if (space.isMemorySpace()) {
AddressIndexKeyIterator addressKeyIterator = new AddressIndexKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, space.getMinAddress(), space.getMaxAddress(), false);
V3_SYMBOL_ADDR_COL, addrMap, space.getMinAddress(), space.getMaxAddress(), false);
if (addressKeyIterator.hasNext()) {
return addrMap.decodeAddress(addressKeyIterator.next());
}
@@ -323,7 +323,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
else {
LongField max = new LongField(addrMap.getKey(space.getMaxAddress(), false));
DBFieldIterator iterator =
symbolTable.indexFieldIterator(null, max, false, SYMBOL_ADDR_COL);
symbolTable.indexFieldIterator(null, max, false, V3_SYMBOL_ADDR_COL);
if (iterator.hasPrevious()) {
LongField val = (LongField) iterator.previous();
Address addr = addrMap.decodeAddress(val.getLongValue());
@@ -341,8 +341,8 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
}
/**
* Returns a record matching the current database schema from the version 2 record.
* @param record the record matching the version 2 schema.
* Returns a record matching the current database schema from the version 3 record.
* @param record the record matching the version 3 schema.
* @return a current symbol record.
*/
private DBRecord convertV3Record(DBRecord record) {

300
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/SymbolDatabaseAdapterV4.java
View File

@@ -16,13 +16,11 @@
package ghidra.program.database.symbol;
import java.io.IOException;
import java.util.HashSet;
import java.util.Set;
import db.*;
import ghidra.program.database.map.*;
import ghidra.program.database.util.EmptyRecordIterator;
import ghidra.program.database.util.RecordFilter;
import ghidra.program.model.address.*;
import ghidra.program.model.symbol.SourceType;
import ghidra.program.model.symbol.SymbolType;
@@ -31,7 +29,7 @@ import ghidra.util.exception.VersionException;
import ghidra.util.task.TaskMonitor;
/**
* SymbolDatabaseAdapter for version 3
* SymbolDatabaseAdapter for version 4
*
* This version added additional sparse columns to store optional data specific to certain
* symbol types. The adhoc string data column was eliminated.
@@ -48,94 +46,64 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
// allows us to index this field and quickly find the primary symbols. The field is sparse
// so that non-primary symbols don't consume any space for this field.
static final Schema V4_SYMBOL_SCHEMA = new Schema(SYMBOL_VERSION, "Key",
new Field[] { StringField.INSTANCE, LongField.INSTANCE, LongField.INSTANCE,
ByteField.INSTANCE, ByteField.INSTANCE, LongField.INSTANCE, LongField.INSTANCE,
LongField.INSTANCE, IntField.INSTANCE, StringField.INSTANCE, StringField.INSTANCE,
StringField.INSTANCE, StringField.INSTANCE },
new String[] { "Name", "Address", "Namespace", "Symbol Type", "Flags", "Locator Hash",
"Primary", "Datatype", "Variable Offset", "ExtOrigImportName", "ExtProgAddr", "Comment",
"LibPath" },
new int[] { SYMBOL_HASH_COL, SYMBOL_PRIMARY_COL, SYMBOL_DATATYPE_COL, SYMBOL_VAROFFSET_COL,
SYMBOL_ORIGINAL_IMPORTED_NAME_COL, SYMBOL_EXTERNAL_PROG_ADDR_COL, SYMBOL_COMMENT_COL,
SYMBOL_LIBPATH_COL });
/* Do not remove the following commented out schema! It shows the version 4 symbol table schema. */
// static final Schema V4_SYMBOL_SCHEMA = new Schema(SYMBOL_VERSION, "Key",
// new Field[] { StringField.INSTANCE, LongField.INSTANCE, LongField.INSTANCE,
// ByteField.INSTANCE, ByteField.INSTANCE, LongField.INSTANCE, LongField.INSTANCE,
// LongField.INSTANCE, IntField.INSTANCE, StringField.INSTANCE, StringField.INSTANCE,
// StringField.INSTANCE, StringField.INSTANCE },
// new String[] { "Name", "Address", "Namespace", "Symbol Type", "Flags", "Locator Hash",
// "Primary", "Datatype", "Variable Offset", "ExtOrigImportName", "ExtProgAddr", "Comment",
// "LibPath" },
// new int[] { SYMBOL_HASH_COL, SYMBOL_PRIMARY_COL, SYMBOL_DATATYPE_COL, SYMBOL_VAROFFSET_COL,
// SYMBOL_ORIGINAL_IMPORTED_NAME_COL, SYMBOL_EXTERNAL_PROG_ADDR_COL, SYMBOL_COMMENT_COL,
// SYMBOL_LIBPATH_COL });
static final int V4_SYMBOL_NAME_COL = 0;
static final int V4_SYMBOL_ADDR_COL = 1;
static final int V4_SYMBOL_PARENT_ID_COL = 2;
static final int V4_SYMBOL_TYPE_COL = 3;
static final int V4_SYMBOL_FLAGS_COL = 4;
// Sparse fields - the following fields are not always applicable so they are optional and
// don't consume space in the database if they aren't used.
static final int V4_SYMBOL_HASH_COL = 5;
static final int V4_SYMBOL_PRIMARY_COL = 6;
static final int V4_SYMBOL_DATATYPE_COL = 7;
static final int V4_SYMBOL_VAROFFSET_COL = 8;
static final int V4_SYMBOL_ORIGINAL_IMPORTED_NAME_COL = 9;
static final int V4_SYMBOL_EXTERNAL_PROG_ADDR_COL = 10;
static final int V4_SYMBOL_COMMENT_COL = 11;
static final int V4_SYMBOL_LIBPATH_COL = 12;
private Table symbolTable;
private AddressMap addrMap;
SymbolDatabaseAdapterV4(DBHandle handle, AddressMap addrMap, boolean create)
throws VersionException, IOException {
SymbolDatabaseAdapterV4(DBHandle handle, AddressMap addrMap) throws VersionException {
this.addrMap = addrMap;
if (create) {
symbolTable = handle.createTable(SYMBOL_TABLE_NAME, SYMBOL_SCHEMA,
new int[] { SYMBOL_ADDR_COL, SYMBOL_NAME_COL, SYMBOL_PARENT_ID_COL, SYMBOL_HASH_COL,
SYMBOL_PRIMARY_COL, SYMBOL_ORIGINAL_IMPORTED_NAME_COL,
SYMBOL_EXTERNAL_PROG_ADDR_COL });
symbolTable = handle.getTable(SYMBOL_TABLE_NAME);
if (symbolTable == null) {
throw new VersionException("Missing Table: " + SYMBOL_TABLE_NAME);
}
else {
symbolTable = handle.getTable(SYMBOL_TABLE_NAME);
if (symbolTable == null) {
throw new VersionException("Missing Table: " + SYMBOL_TABLE_NAME);
}
if (symbolTable.getSchema().getVersion() != SYMBOL_VERSION) {
int version = symbolTable.getSchema().getVersion();
if (version < SYMBOL_VERSION) {
throw new VersionException(true);
}
throw new VersionException(VersionException.NEWER_VERSION, false);
if (symbolTable.getSchema().getVersion() != SYMBOL_VERSION) {
int version = symbolTable.getSchema().getVersion();
if (version < SYMBOL_VERSION) {
throw new VersionException(true);
}
throw new VersionException(VersionException.NEWER_VERSION, false);
}
}
// @Override
// DBRecord createSymbol(String name, Address address, long namespaceID, SymbolType symbolType,
// SourceType source, boolean isPrimary) throws IOException {
// long nextID = symbolTable.getKey();
//
// // avoiding key 0, as it is reserved for the global namespace
// if (nextID == 0) {
// nextID++;
// }
// return createSymbol(nextID, name, address, namespaceID, symbolType, (byte) source.getStorageId(),
// isPrimary);
// }
@Override
DBRecord createSymbolRecord(String name, long namespaceID, Address address,
SymbolType symbolType, boolean isPrimary, SourceType source) {
long nextID = symbolTable.getKey();
// Avoid key 0, as it is reserved for the global namespace
if (nextID == 0) {
nextID++;
}
long addressKey = addrMap.getKey(address, true);
DBRecord rec = symbolTable.getSchema().createRecord(nextID);
rec.setString(SYMBOL_NAME_COL, name);
rec.setLongValue(SYMBOL_ADDR_COL, addressKey);
rec.setLongValue(SYMBOL_PARENT_ID_COL, namespaceID);
rec.setByteValue(SYMBOL_TYPE_COL, symbolType.getID());
rec.setByteValue(SYMBOL_FLAGS_COL, getSourceTypeFlagsBits(source)); // assume non-pinned
// Sparse columns - these columns don't apply to all symbols.
// they default to null unless specifically set. Null values don't consume space.
rec.setField(SYMBOL_HASH_COL, computeLocatorHash(name, namespaceID, addressKey));
if (isPrimary) {
rec.setLongValue(SYMBOL_PRIMARY_COL, addressKey);
}
return rec;
throw new UnsupportedOperationException();
}
@Override
void removeSymbol(long symbolID) throws IOException {
symbolTable.deleteRecord(symbolID);
throw new UnsupportedOperationException();
}
@Override
@@ -144,7 +112,7 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) {
return false;
}
return symbolTable.hasRecord(new LongField(key), SYMBOL_ADDR_COL);
return symbolTable.hasRecord(new LongField(key), V4_SYMBOL_ADDR_COL);
}
@Override
@@ -153,12 +121,12 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) {
return Field.EMPTY_ARRAY;
}
return symbolTable.findRecords(new LongField(key), SYMBOL_ADDR_COL);
return symbolTable.findRecords(new LongField(key), V4_SYMBOL_ADDR_COL);
}
@Override
DBRecord getSymbolRecord(long symbolID) throws IOException {
return symbolTable.getRecord(symbolID);
return convertV4Record(symbolTable.getRecord(symbolID));
}
@Override
@@ -168,116 +136,117 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
@Override
RecordIterator getSymbolsByAddress(boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable,
new AddressIndexPrimaryKeyIterator(symbolTable, SYMBOL_ADDR_COL, addrMap, forward));
KeyToRecordIterator it = new KeyToRecordIterator(symbolTable,
new AddressIndexPrimaryKeyIterator(symbolTable, V4_SYMBOL_ADDR_COL, addrMap, forward));
return new V4ConvertedRecordIterator(it);
}
@Override
RecordIterator getSymbolsByAddress(Address startAddr, boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, startAddr, forward));
KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
V4_SYMBOL_ADDR_COL, addrMap, startAddr, forward));
return new V4ConvertedRecordIterator(it);
}
@Override
void updateSymbolRecord(DBRecord record) throws IOException {
// make sure hash is updated to current name and name space
String name = record.getString(SYMBOL_NAME_COL);
long namespaceId = record.getLongValue(SYMBOL_PARENT_ID_COL);
long addressKey = record.getLongValue(SYMBOL_ADDR_COL);
record.setField(SYMBOL_HASH_COL, computeLocatorHash(name, namespaceId, addressKey));
symbolTable.putRecord(record);
throw new UnsupportedOperationException();
}
@Override
RecordIterator getSymbols() throws IOException {
return symbolTable.iterator();
return new V4ConvertedRecordIterator(symbolTable.iterator());
}
@Override
RecordIterator getSymbols(Address start, Address end, boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, start, end, forward));
KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
V4_SYMBOL_ADDR_COL, addrMap, start, end, forward));
return new V4ConvertedRecordIterator(it);
}
@Override
RecordIterator getSymbols(AddressSetView set, boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, set, forward));
KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
V4_SYMBOL_ADDR_COL, addrMap, set, forward));
return new V4ConvertedRecordIterator(it);
}
@Override
protected RecordIterator getPrimarySymbols(AddressSetView set, boolean forward)
throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_PRIMARY_COL, addrMap, set, forward));
KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
V4_SYMBOL_PRIMARY_COL, addrMap, set, forward));
return new V4ConvertedRecordIterator(it);
}
@Override
protected DBRecord getPrimarySymbol(Address address) throws IOException {
AddressIndexPrimaryKeyIterator it = new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_PRIMARY_COL, addrMap, address, address, true);
V4_SYMBOL_PRIMARY_COL, addrMap, address, address, true);
if (it.hasNext()) {
return symbolTable.getRecord(it.next());
return convertV4Record(symbolTable.getRecord(it.next()));
}
return null;
}
void deleteExternalEntries(Address start, Address end) throws IOException {
AddressRecordDeleter.deleteRecords(symbolTable, SYMBOL_ADDR_COL, addrMap, start, end, null);
throw new UnsupportedOperationException();
}
@Override
void moveAddress(Address oldAddr, Address newAddr) throws IOException {
LongField oldKey = new LongField(addrMap.getKey(oldAddr, false));
long newKey = addrMap.getKey(newAddr, true);
Field[] keys = symbolTable.findRecords(oldKey, SYMBOL_ADDR_COL);
for (Field key : keys) {
DBRecord rec = symbolTable.getRecord(key);
rec.setLongValue(SYMBOL_ADDR_COL, newKey);
symbolTable.putRecord(rec);
}
throw new UnsupportedOperationException();
}
@Override
Set<Address> deleteAddressRange(Address startAddr, Address endAddr, TaskMonitor monitor)
throws CancelledException, IOException {
AnchoredSymbolRecordFilter filter = new AnchoredSymbolRecordFilter();
AddressRecordDeleter.deleteRecords(symbolTable, SYMBOL_ADDR_COL, addrMap, startAddr,
endAddr, filter);
return filter.getAddressesForSkippedRecords();
throw new UnsupportedOperationException();
}
@Override
RecordIterator getSymbolsByNamespace(long id) throws IOException {
LongField field = new LongField(id);
return symbolTable.indexIterator(SYMBOL_PARENT_ID_COL, field, field, true);
RecordIterator it =
symbolTable.indexIterator(V4_SYMBOL_PARENT_ID_COL, field, field, true);
return new V4ConvertedRecordIterator(it);
}
@Override
RecordIterator getSymbolsByName(String name) throws IOException {
StringField field = new StringField(name);
return symbolTable.indexIterator(SYMBOL_NAME_COL, field, field, true);
RecordIterator it = symbolTable.indexIterator(V4_SYMBOL_NAME_COL, field, field, true);
return new V4ConvertedRecordIterator(it);
}
@Override
RecordIterator scanSymbolsByName(String startName) throws IOException {
StringField field = new StringField(startName);
return symbolTable.indexIterator(SYMBOL_NAME_COL, field, null, true);
RecordIterator it = symbolTable.indexIterator(V4_SYMBOL_NAME_COL, field, null, true);
return new V4ConvertedRecordIterator(it);
}
@Override
RecordIterator getExternalSymbolsByOriginalImportName(String extLabel) throws IOException {
StringField extLabelField = new StringField(extLabel);
return symbolTable.indexIterator(SYMBOL_ORIGINAL_IMPORTED_NAME_COL, extLabelField,
RecordIterator it =
symbolTable.indexIterator(V4_SYMBOL_ORIGINAL_IMPORTED_NAME_COL, extLabelField,
extLabelField, true);
return new V4ConvertedRecordIterator(it);
}
@Override
RecordIterator getExternalSymbolsByMemoryAddress(Address extProgAddr) throws IOException {
StringField addrField = new StringField(extProgAddr.toString());
return symbolTable.indexIterator(SYMBOL_EXTERNAL_PROG_ADDR_COL, addrField, addrField, true);
RecordIterator it =
symbolTable.indexIterator(V4_SYMBOL_EXTERNAL_PROG_ADDR_COL, addrField, addrField,
true);
return new V4ConvertedRecordIterator(it);
}
@Override
@@ -291,8 +260,9 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
Field end = computeLocatorHash(name, id, MAX_ADDRESS_OFFSET);
RecordIterator it = symbolTable.indexIterator(SYMBOL_HASH_COL, start, end, true);
return getNameAndNamespaceFilterIterator(name, id, it);
RecordIterator it = symbolTable.indexIterator(V4_SYMBOL_HASH_COL, start, end, true);
it = getNameAndNamespaceFilterIterator(name, id, it);
return new V4ConvertedRecordIterator(it);
}
@Override
@@ -302,11 +272,11 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
if (search == null) {
return null;
}
RecordIterator it = symbolTable.indexIterator(SYMBOL_HASH_COL, search, search, true);
RecordIterator it = symbolTable.indexIterator(V4_SYMBOL_HASH_COL, search, search, true);
RecordIterator filtered =
getNameNamespaceAddressFilterIterator(name, namespaceId, addressKey, it);
if (filtered.hasNext()) {
return filtered.next();
return convertV4Record(filtered.next());
}
return null;
}
@@ -315,7 +285,7 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
Address getMaxSymbolAddress(AddressSpace space) throws IOException {
if (space.isMemorySpace()) {
AddressIndexKeyIterator addressKeyIterator = new AddressIndexKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, space.getMinAddress(), space.getMaxAddress(), false);
V4_SYMBOL_ADDR_COL, addrMap, space.getMinAddress(), space.getMaxAddress(), false);
if (addressKeyIterator.hasNext()) {
return addrMap.decodeAddress(addressKeyIterator.next());
}
@@ -323,7 +293,7 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
else {
LongField max = new LongField(addrMap.getKey(space.getMaxAddress(), false));
DBFieldIterator iterator =
symbolTable.indexFieldIterator(null, max, false, SYMBOL_ADDR_COL);
symbolTable.indexFieldIterator(null, max, false, V4_SYMBOL_ADDR_COL);
if (iterator.hasPrevious()) {
LongField val = (LongField) iterator.previous();
Address addr = addrMap.decodeAddress(val.getLongValue());
@@ -340,24 +310,82 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
return symbolTable;
}
private class AnchoredSymbolRecordFilter implements RecordFilter {
private Set<Address> set = new HashSet<Address>();
/**
* Returns a record matching the current database schema from the version 4 record.
* @param record the record matching the version 4 schema.
* @return a current symbol record.
*/
private DBRecord convertV4Record(DBRecord record) {
if (record == null) {
return null;
}
DBRecord rec = SymbolDatabaseAdapter.SYMBOL_SCHEMA.createRecord(record.getKey());
@Override
public boolean matches(DBRecord record) {
// only move symbols whose anchor flag is not on
Address addr = addrMap.decodeAddress(record.getLongValue(SYMBOL_ADDR_COL));
byte flags = record.getByteValue(SymbolDatabaseAdapter.SYMBOL_FLAGS_COL);
boolean pinned = (flags & SymbolDatabaseAdapter.SYMBOL_PINNED_FLAG) != 0;
if (!pinned) {
return true;
}
set.add(addr);
return false;
String symbolName = record.getString(V4_SYMBOL_NAME_COL);
rec.setString(SymbolDatabaseAdapter.SYMBOL_NAME_COL, symbolName);
long symbolAddrKey = record.getLongValue(V4_SYMBOL_ADDR_COL);
rec.setLongValue(SymbolDatabaseAdapter.SYMBOL_ADDR_COL, symbolAddrKey);
long namespaceId = record.getLongValue(V4_SYMBOL_PARENT_ID_COL);
rec.setLongValue(SymbolDatabaseAdapter.SYMBOL_PARENT_ID_COL, namespaceId);
byte symbolTypeId = record.getByteValue(V4_SYMBOL_TYPE_COL);
rec.setByteValue(SymbolDatabaseAdapter.SYMBOL_TYPE_COL, symbolTypeId);
rec.setByteValue(SymbolDatabaseAdapter.SYMBOL_FLAGS_COL,
record.getByteValue(V4_SYMBOL_FLAGS_COL));
//
// Convert sparse columns
//
if (symbolTypeId == SYMBOL_TYPE_LABEL || symbolTypeId == SYMBOL_TYPE_FUNCTION) {
record.setString(SYMBOL_EXTERNAL_PROG_ADDR_COL,
record.getString(V4_SYMBOL_EXTERNAL_PROG_ADDR_COL));
record.setString(SYMBOL_ORIGINAL_IMPORTED_NAME_COL,
record.getString(V4_SYMBOL_ORIGINAL_IMPORTED_NAME_COL));
}
else if (symbolTypeId == SYMBOL_TYPE_LOCAL_VAR || symbolTypeId == SYMBOL_TYPE_PARAMETER) {
record.setString(SYMBOL_COMMENT_COL, record.getString(V4_SYMBOL_COMMENT_COL));
}
else if (symbolTypeId == SYMBOL_TYPE_LIBRARY) {
// NOTE: don't set new sparse ordinal column
record.setString(SYMBOL_LIBPATH_COL, record.getString(V4_SYMBOL_LIBPATH_COL));
}
Set<Address> getAddressesForSkippedRecords() {
return set;
Field hash = record.getFieldValue(V4_SYMBOL_HASH_COL);
if (hash != null) {
rec.setField(SymbolDatabaseAdapter.SYMBOL_HASH_COL, hash);
}
Field primaryAddr = record.getFieldValue(V4_SYMBOL_PRIMARY_COL);
if (primaryAddr != null) {
rec.setField(SymbolDatabaseAdapter.SYMBOL_PRIMARY_COL, primaryAddr);
}
Field dataTypeId = record.getFieldValue(V4_SYMBOL_DATATYPE_COL);
if (dataTypeId != null) {
rec.setField(SymbolDatabaseAdapter.SYMBOL_DATATYPE_COL, dataTypeId);
}
Field varOffset = record.getFieldValue(V4_SYMBOL_VAROFFSET_COL);
if (varOffset != null) {
rec.setField(SymbolDatabaseAdapter.SYMBOL_VAROFFSET_COL, varOffset);
}
return rec;
}
private class V4ConvertedRecordIterator extends ConvertedRecordIterator {
V4ConvertedRecordIterator(RecordIterator originalIterator) {
super(originalIterator, false);
}
@Override
protected DBRecord convertRecord(DBRecord record) {
return convertV4Record(record);
}
}

351
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/SymbolDatabaseAdapterV5.java Normal file
View File

@@ -0,0 +1,351 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.program.database.symbol;
import java.io.IOException;
import java.util.HashSet;
import java.util.Set;
import db.*;
import ghidra.program.database.map.*;
import ghidra.program.database.util.EmptyRecordIterator;
import ghidra.program.database.util.RecordFilter;
import ghidra.program.model.address.*;
import ghidra.program.model.symbol.SourceType;
import ghidra.program.model.symbol.SymbolType;
import ghidra.util.exception.CancelledException;
import ghidra.util.exception.VersionException;
import ghidra.util.task.TaskMonitor;
/**
* SymbolDatabaseAdapter for version 5
*
* This version added additional sparse columns to store optional data specific to certain
* symbol types. The adhoc string data column was eliminated.
*/
class SymbolDatabaseAdapterV5 extends SymbolDatabaseAdapter {
static final int SYMBOL_VERSION = 5;
// Used to create a range when searching symbols by name/namespace but don't care about address
private static final long MIN_ADDRESS_OFFSET = 0;
private static final long MAX_ADDRESS_OFFSET = -1;
// NOTE: the primary field duplicates the symbol's address when the symbol is primary. This
// allows us to index this field and quickly find the primary symbols. The field is sparse
// so that non-primary symbols don't consume any space for this field.
static final Schema V5_SYMBOL_SCHEMA = new Schema(SYMBOL_VERSION, "Key",
new Field[] { StringField.INSTANCE, LongField.INSTANCE, LongField.INSTANCE,
ByteField.INSTANCE, ByteField.INSTANCE, LongField.INSTANCE, LongField.INSTANCE,
LongField.INSTANCE, IntField.INSTANCE, StringField.INSTANCE, StringField.INSTANCE,
StringField.INSTANCE, StringField.INSTANCE, IntField.INSTANCE },
new String[] { "Name", "Address", "Namespace", "Symbol Type", "Flags", "Locator Hash",
"Primary", "Datatype", "Variable Offset", "ExtOrigImportName", "ExtProgAddr", "Comment",
"LibPath", "LibOrdinal" },
new int[] { SYMBOL_HASH_COL, SYMBOL_PRIMARY_COL, SYMBOL_DATATYPE_COL, SYMBOL_VAROFFSET_COL,
SYMBOL_ORIGINAL_IMPORTED_NAME_COL, SYMBOL_EXTERNAL_PROG_ADDR_COL, SYMBOL_COMMENT_COL,
SYMBOL_LIBPATH_COL, SYMBOL_LIB_ORDINAL_COL });
private Table symbolTable;
private AddressMap addrMap;
SymbolDatabaseAdapterV5(DBHandle handle, AddressMap addrMap, boolean create)
throws VersionException, IOException {
this.addrMap = addrMap;
if (create) {
symbolTable = handle.createTable(SYMBOL_TABLE_NAME, SYMBOL_SCHEMA,
new int[] { SYMBOL_ADDR_COL, SYMBOL_NAME_COL, SYMBOL_PARENT_ID_COL, SYMBOL_HASH_COL,
SYMBOL_PRIMARY_COL, SYMBOL_ORIGINAL_IMPORTED_NAME_COL,
SYMBOL_EXTERNAL_PROG_ADDR_COL });
}
else {
symbolTable = handle.getTable(SYMBOL_TABLE_NAME);
if (symbolTable == null) {
throw new VersionException("Missing Table: " + SYMBOL_TABLE_NAME);
}
if (symbolTable.getSchema().getVersion() != SYMBOL_VERSION) {
int version = symbolTable.getSchema().getVersion();
if (version < SYMBOL_VERSION) {
throw new VersionException(true);
}
throw new VersionException(VersionException.NEWER_VERSION, false);
}
}
}
@Override
DBRecord createSymbolRecord(String name, long namespaceID, Address address,
SymbolType symbolType, boolean isPrimary, SourceType source) {
long nextID = symbolTable.getKey();
// Avoid key 0, as it is reserved for the global namespace
if (nextID == 0) {
nextID++;
}
long addressKey = addrMap.getKey(address, true);
DBRecord rec = symbolTable.getSchema().createRecord(nextID);
rec.setString(SYMBOL_NAME_COL, name);
rec.setLongValue(SYMBOL_ADDR_COL, addressKey);
rec.setLongValue(SYMBOL_PARENT_ID_COL, namespaceID);
rec.setByteValue(SYMBOL_TYPE_COL, symbolType.getID());
rec.setByteValue(SYMBOL_FLAGS_COL, getSourceTypeFlagsBits(source)); // assume non-pinned
// Sparse columns - these columns don't apply to all symbols.
// they default to null unless specifically set. Null values don't consume space.
rec.setField(SYMBOL_HASH_COL, computeLocatorHash(name, namespaceID, addressKey));
if (isPrimary) {
rec.setLongValue(SYMBOL_PRIMARY_COL, addressKey);
}
return rec;
}
@Override
void removeSymbol(long symbolID) throws IOException {
symbolTable.deleteRecord(symbolID);
}
@Override
boolean hasSymbol(Address addr) throws IOException {
long key = addrMap.getKey(addr, false);
if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) {
return false;
}
return symbolTable.hasRecord(new LongField(key), SYMBOL_ADDR_COL);
}
@Override
Field[] getSymbolIDs(Address addr) throws IOException {
long key = addrMap.getKey(addr, false);
if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) {
return Field.EMPTY_ARRAY;
}
return symbolTable.findRecords(new LongField(key), SYMBOL_ADDR_COL);
}
@Override
DBRecord getSymbolRecord(long symbolID) throws IOException {
return symbolTable.getRecord(symbolID);
}
@Override
int getSymbolCount() {
return symbolTable.getRecordCount();
}
@Override
RecordIterator getSymbolsByAddress(boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable,
new AddressIndexPrimaryKeyIterator(symbolTable, SYMBOL_ADDR_COL, addrMap, forward));
}
@Override
RecordIterator getSymbolsByAddress(Address startAddr, boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, startAddr, forward));
}
@Override
void updateSymbolRecord(DBRecord record) throws IOException {
// make sure hash is updated to current name and name space
String name = record.getString(SYMBOL_NAME_COL);
long namespaceId = record.getLongValue(SYMBOL_PARENT_ID_COL);
long addressKey = record.getLongValue(SYMBOL_ADDR_COL);
record.setField(SYMBOL_HASH_COL, computeLocatorHash(name, namespaceId, addressKey));
symbolTable.putRecord(record);
}
@Override
RecordIterator getSymbols() throws IOException {
return symbolTable.iterator();
}
@Override
RecordIterator getSymbols(Address start, Address end, boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, start, end, forward));
}
@Override
RecordIterator getSymbols(AddressSetView set, boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, set, forward));
}
@Override
protected RecordIterator getPrimarySymbols(AddressSetView set, boolean forward)
throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_PRIMARY_COL, addrMap, set, forward));
}
@Override
protected DBRecord getPrimarySymbol(Address address) throws IOException {
AddressIndexPrimaryKeyIterator it = new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_PRIMARY_COL, addrMap, address, address, true);
if (it.hasNext()) {
return symbolTable.getRecord(it.next());
}
return null;
}
void deleteExternalEntries(Address start, Address end) throws IOException {
AddressRecordDeleter.deleteRecords(symbolTable, SYMBOL_ADDR_COL, addrMap, start, end, null);
}
@Override
void moveAddress(Address oldAddr, Address newAddr) throws IOException {
LongField oldKey = new LongField(addrMap.getKey(oldAddr, false));
long newKey = addrMap.getKey(newAddr, true);
Field[] keys = symbolTable.findRecords(oldKey, SYMBOL_ADDR_COL);
for (Field key : keys) {
DBRecord rec = symbolTable.getRecord(key);
rec.setLongValue(SYMBOL_ADDR_COL, newKey);
symbolTable.putRecord(rec);
}
}
@Override
Set<Address> deleteAddressRange(Address startAddr, Address endAddr, TaskMonitor monitor)
throws CancelledException, IOException {
AnchoredSymbolRecordFilter filter = new AnchoredSymbolRecordFilter();
AddressRecordDeleter.deleteRecords(symbolTable, SYMBOL_ADDR_COL, addrMap, startAddr,
endAddr, filter);
return filter.getAddressesForSkippedRecords();
}
@Override
RecordIterator getSymbolsByNamespace(long id) throws IOException {
LongField field = new LongField(id);
return symbolTable.indexIterator(SYMBOL_PARENT_ID_COL, field, field, true);
}
@Override
RecordIterator getSymbolsByName(String name) throws IOException {
StringField field = new StringField(name);
return symbolTable.indexIterator(SYMBOL_NAME_COL, field, field, true);
}
@Override
RecordIterator scanSymbolsByName(String startName) throws IOException {
StringField field = new StringField(startName);
return symbolTable.indexIterator(SYMBOL_NAME_COL, field, null, true);
}
@Override
RecordIterator getExternalSymbolsByOriginalImportName(String extLabel) throws IOException {
StringField extLabelField = new StringField(extLabel);
return symbolTable.indexIterator(SYMBOL_ORIGINAL_IMPORTED_NAME_COL, extLabelField,
extLabelField, true);
}
@Override
RecordIterator getExternalSymbolsByMemoryAddress(Address extProgAddr) throws IOException {
StringField addrField = new StringField(extProgAddr.toString());
return symbolTable.indexIterator(SYMBOL_EXTERNAL_PROG_ADDR_COL, addrField, addrField, true);
}
@Override
RecordIterator getSymbolsByNameAndNamespace(String name, long id) throws IOException {
// create a range of hash fields for all symbols with this name and namespace id over all
// possible addresses
Field start = computeLocatorHash(name, id, MIN_ADDRESS_OFFSET);
if (start == null) {
return EmptyRecordIterator.INSTANCE;
}
Field end = computeLocatorHash(name, id, MAX_ADDRESS_OFFSET);
RecordIterator it = symbolTable.indexIterator(SYMBOL_HASH_COL, start, end, true);
return getNameAndNamespaceFilterIterator(name, id, it);
}
@Override
DBRecord getSymbolRecord(Address address, String name, long namespaceId) throws IOException {
long addressKey = addrMap.getKey(address, false);
Field search = computeLocatorHash(name, namespaceId, addressKey);
if (search == null) {
return null;
}
RecordIterator it = symbolTable.indexIterator(SYMBOL_HASH_COL, search, search, true);
RecordIterator filtered =
getNameNamespaceAddressFilterIterator(name, namespaceId, addressKey, it);
if (filtered.hasNext()) {
return filtered.next();
}
return null;
}
@Override
Address getMaxSymbolAddress(AddressSpace space) throws IOException {
if (space.isMemorySpace()) {
AddressIndexKeyIterator addressKeyIterator = new AddressIndexKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, space.getMinAddress(), space.getMaxAddress(), false);
if (addressKeyIterator.hasNext()) {
return addrMap.decodeAddress(addressKeyIterator.next());
}
}
else {
LongField max = new LongField(addrMap.getKey(space.getMaxAddress(), false));
DBFieldIterator iterator =
symbolTable.indexFieldIterator(null, max, false, SYMBOL_ADDR_COL);
if (iterator.hasPrevious()) {
LongField val = (LongField) iterator.previous();
Address addr = addrMap.decodeAddress(val.getLongValue());
if (space.equals(addr.getAddressSpace())) {
return addr;
}
}
}
return null;
}
@Override
Table getTable() {
return symbolTable;
}
private class AnchoredSymbolRecordFilter implements RecordFilter {
private Set<Address> set = new HashSet<Address>();
@Override
public boolean matches(DBRecord record) {
// only move symbols whose anchor flag is not on
Address addr = addrMap.decodeAddress(record.getLongValue(SYMBOL_ADDR_COL));
byte flags = record.getByteValue(SymbolDatabaseAdapter.SYMBOL_FLAGS_COL);
boolean pinned = (flags & SymbolDatabaseAdapter.SYMBOL_PINNED_FLAG) != 0;
if (!pinned) {
return true;
}
set.add(addr);
return false;
}
Set<Address> getAddressesForSkippedRecords() {
return set;
}
}
}

191
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/SymbolManager.java
View File

@@ -72,6 +72,8 @@ public class SymbolManager implements SymbolTable, ManagerDB {
private NamespaceManager namespaceMgr;
private VariableStorageManagerDB variableStorageMgr;
private List<LibrarySymbol> libSymbolList; // unmodifiable; use getLibrarySymbolList() to obtain instance
private OldVariableStorageManagerDB oldVariableStorageMgr; // required for upgrade
private AddressMapImpl dynamicSymbolAddressMap;
@@ -182,6 +184,10 @@ public class SymbolManager implements SymbolTable, ManagerDB {
// older than program version 10
processOldVariableAddresses(monitor);
}
if (currentRevision < ProgramDB.LIBRARY_ORDINAL_ASSIGNMENT_ADDED_VERSION) {
assignLibraryOrdinals();
}
}
}
@@ -663,9 +669,6 @@ public class SymbolManager implements SymbolTable, ManagerDB {
SymbolType symType = sym.getSymbolType();
try {
Address address = sym.getAddress();
// if (address.isVariableAddress()) {
// variableStorageMgr.deleteVariableStorage(address);
// }
String name = sym.getName();
boolean primary = sym.isPrimary();
@@ -674,7 +677,6 @@ public class SymbolManager implements SymbolTable, ManagerDB {
adapter.removeSymbol(id);
cache.delete(id);
//sym.setInvalid(); // already invalidated by removeObj
// if any symbols still exist here, then
// make one of these remaining symbols 'primary'
@@ -734,11 +736,10 @@ public class SymbolManager implements SymbolTable, ManagerDB {
private SymbolDB getDynamicSymbol(Address memoryAddress) {
long symbolID = getDynamicSymbolID(memoryAddress);
// retrieve the symbol from the cache without validating or refreshing it. We know
// if it exists is should
// Retrieve the symbol from the cache without validating or refreshing it.
CodeSymbol symbol = (CodeSymbol) cache.getRaw(symbolID);
if (symbol != null) {
// we know if we got here, the symbol is valid regardless of what it thinks
// We know if we got here the symbol is valid, so force it to be valid
symbol.setIsValid();
return symbol;
}
@@ -1378,8 +1379,8 @@ public class SymbolManager implements SymbolTable, ManagerDB {
@Override
public LabelHistory[] getLabelHistory(Address addr) {
ArrayList<LabelHistory> list = new ArrayList<>();
try {
try (Closeable c = lock.read()) {
ArrayList<LabelHistory> list = new ArrayList<>();
RecordIterator iter = historyAdapter.getRecordsByAddress(addrMap.getKey(addr, false));
while (iter.hasNext()) {
DBRecord rec = iter.next();
@@ -1401,8 +1402,9 @@ public class SymbolManager implements SymbolTable, ManagerDB {
@Override
public void invalidateCache(boolean all) {
variableStorageMgr.invalidateCache(all);
try (Closeable c = lock.write()) {
variableStorageMgr.invalidateCache(all);
libSymbolList = null;
cache.invalidate();
dynamicSymbolAddressMap.reconcile();
}
@@ -2739,7 +2741,11 @@ public class SymbolManager implements SymbolTable, ManagerDB {
}
/**
* Create a Library symbol with the specified name and optional pathname
* Create a Library symbol with the specified name and optional pathname.
* The new Library symbol will be assigned the next available ordinal at the end of the
* ordered Library symbol sequence. Use {@link LibrarySymbol#setOrdinal(int)} to adjust
* ordinal.
*
*
* @param name library name
* @param pathname project file path (may be null)
@@ -2757,14 +2763,24 @@ public class SymbolManager implements SymbolTable, ManagerDB {
try (Closeable c = lock.write()) {
DBRecord record = doCreateBasicSymbolRecord(name, null, Address.NO_ADDRESS,
SymbolType.LIBRARY, false, source, true);
LibrarySymbol.setRecordFields(record, pathname);
List<LibrarySymbol> librarySymbolList = new ArrayList<>(getLibrarySymbolList());
int ordinal = librarySymbolList.size();
if (Library.UNKNOWN.equals(name)) {
// UNKNOWN Library always assigned ordinal of 0
ordinal = 0;
}
LibrarySymbol newLibSymbol = new LibrarySymbol(this, record);
LibrarySymbol.setRecordFields(record, ordinal, pathname);
adapter.updateSymbolRecord(record);
cache.add(newLibSymbol);
LibrarySymbol newSymbol = new LibrarySymbol(this, record);
cache.add(newSymbol);
symbolAdded(newSymbol);
return newSymbol;
librarySymbolList.add(ordinal, newLibSymbol);
assignLibraryOrdinals(librarySymbolList, true);
symbolAdded(newLibSymbol);
return newLibSymbol;
}
catch (IOException e) {
program.dbError(e); // will not return
@@ -2772,6 +2788,139 @@ public class SymbolManager implements SymbolTable, ManagerDB {
}
}
/**
* {@return computed ordinal of the specified library symbol}
* @param libSym library symbol
*/
int computeLibraryOrdinal(LibrarySymbol libSym) {
List<LibrarySymbol> list = getLibrarySymbolList();
for (int i = 0; i < list.size(); i++) {
LibrarySymbol s = list.get(i);
if (s == libSym) {
return i;
}
}
throw new AssertionError("Invalid symbol instance");
}
/**
* @return an ordered unmodifiable list of Library symbols
*/
public List<LibrarySymbol> getLibrarySymbolList() {
List<LibrarySymbol> list = libSymbolList;
if (list == null) {
try (Closeable c = lock.read()) {
list = buildLibrarySymbolList();
libSymbolList = list;
}
catch (IOException e) {
dbError(e);
}
}
return list;
}
private List<LibrarySymbol> buildLibrarySymbolList() throws IOException {
List<LibrarySymbol> list = new ArrayList<>();
byte libraryTypeId = SymbolType.LIBRARY.getID();
RecordIterator iter = new QueryRecordIterator(adapter.getSymbols(),
rec -> libraryTypeId == rec
.getByteValue(SymbolDatabaseAdapter.SYMBOL_TYPE_COL));
// Assume records will be returned in order of symbol ID
long lastId = -1;
while (iter.hasNext()) {
LibrarySymbol libSym = (LibrarySymbol) getSymbol(iter.next());
long id = libSym.getID();
if (id <= lastId) {
throw new AssertionError("Unexpected symbol order");
}
list.add(libSym);
}
Collections.sort(list);
return Collections.unmodifiableList(list);
}
/**
* Assign and store ordinals for all library symbols during upgrade
*/
private void assignLibraryOrdinals() {
List<LibrarySymbol> libSymList = new ArrayList<>(getLibrarySymbolList());
for (int i = 0; i < libSymList.size(); i++) {
LibrarySymbol libSym = libSymList.get(i);
if (Library.UNKNOWN.equals(libSym.getName())) {
// UNKNOWN Library always assigned ordinal of 0 but may have arbitrary
// placement in list prior to upgrade
if (i != 0) {
libSymList.remove(i);
libSymList.add(0, libSym);
}
break;
}
}
assignLibraryOrdinals(libSymList, false);
}
/**
* Update library symbol ordinal to reflect current placement within ordered list.
* The cached {@code libSymbolList} will be updated to refer to the list instance provided.
* @param list new library symbol ordered list
* @param notify if true any library symbol ordinal change will generate change event for
* that symbol.
*/
void assignLibraryOrdinals(List<LibrarySymbol> list, boolean notify) {
for (int i = 0; i < list.size(); i++) {
LibrarySymbol libSym = list.get(i);
libSym.doSetOrdinal(i, notify);
}
libSymbolList = list; // update cached list
}
/**
* Adjust ordinals based upon the movement of an existing Library symbol.
* <p>
* NOTE: The caller is responsible for not displacing UNKNOWN Library if already exists
* at ordinal 0.
*
* @param libSym existing Library symbol
* @param newOrdinal non-negative ordinal (max value will be limited)
*/
void adjustLibraryOrdinals(LibrarySymbol libSym, int newOrdinal) {
if (newOrdinal < 0) {
throw new IllegalArgumentException("Positive ordinal required");
}
List<LibrarySymbol> libSymList = new ArrayList<>(getLibrarySymbolList());
int listSize = libSymList.size();
if (newOrdinal >= listSize) {
newOrdinal = listSize - 1; // do not go beyond end of list
}
int oldOrdinal = libSym.getOrdinal();
if (oldOrdinal == newOrdinal) {
return;
}
LibrarySymbol s = libSymList.remove(oldOrdinal);
if (s != libSym) {
throw new AssertionError("Library symbol ordinal mismatch with ordered list");
}
libSymList.add(newOrdinal, libSym);
assignLibraryOrdinals(libSymList, true);
}
/**
* {@return library symbol at the specified ordinal or null if ordinal is beyond end of list}
* @param ordinal library symbol ordinal
* @throws IndexOutOfBoundsException if a negative ordinal is specified
*/
LibrarySymbol getLibrarySymbolByOrdinal(int ordinal) throws IndexOutOfBoundsException {
List<LibrarySymbol> librarySymbolList = getLibrarySymbolList();
if (ordinal >= librarySymbolList.size()) {
return null;
}
return librarySymbolList.get(ordinal);
}
/**
* Create a Class symbol with the specified name and parent
*
@@ -2875,6 +3024,11 @@ public class SymbolManager implements SymbolTable, ManagerDB {
source = validateSource(source, name, address, symbolType);
name = validateName(name, source);
if ((symbolType == SymbolType.CLASS || symbolType == SymbolType.NAMESPACE) &&
Library.UNKNOWN.equals(name)) {
throw new InvalidInputException(Library.UNKNOWN + " is a reserved Library name");
}
if (checkForDuplicates) {
checkDuplicateSymbolName(address, name, parent, symbolType);
}
@@ -2905,6 +3059,9 @@ public class SymbolManager implements SymbolTable, ManagerDB {
if (!addr.isExternalAddress()) {
throw new IllegalArgumentException("External address required");
}
if (!namespace.isExternal()) {
throw new IllegalArgumentException("External namespace required");
}
if (externalProgramAddress != null && !externalProgramAddress.isLoadedMemoryAddress()) {
throw new IllegalArgumentException("Memory address required for external program");
}

15
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/listing/Library.java
View File

@@ -16,9 +16,10 @@
package ghidra.program.model.listing;
import ghidra.program.model.symbol.*;
import ghidra.util.exception.InvalidInputException;
/**
* Interface for a Library namespace.
* Interface for a Library dependency and namespace.
*/
public interface Library extends Namespace {
@@ -30,10 +31,20 @@ public interface Library extends Namespace {
}
/**
* @return the associated program within the project which corresponds to this library
* @return the associated program file pathname within the project which corresponds to this library.
*/
public String getAssociatedProgramPath();
/**
* Sets the program file pathname within the project which corresponds to this library.
* <p>
* NOTE: Assigning a path to {@link Library#UNKNOWN} Library will be ignored.
*
* @param programPath a program file pathname or null to clear the stored path.
* @throws InvalidInputException on invalid programPath is specified
*/
public void setAssociatedProgramPath(String programPath) throws InvalidInputException;
/**
* Get the Library which contains the specified external symbol.
* @param symbol external symbol

13
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/symbol/ExternalLocation.java
View File

@@ -4,9 +4,9 @@
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
*
* http://www.apache.org/licenses/LICENSE-2.0
*
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -42,6 +42,15 @@ public interface ExternalLocation {
*/
public String getLibraryName();
/**
* NOTE: If this external location corresponds to a back-reference this
* may correspond to an application and not a library.
*
* @return the external Program path which contains the referenced symbol or
* null if unknown.
*/
public String getExternalLibraryPath();
/**
* Returns the parent namespace containing this location.
* @return the parent namespace containing this location.

74
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/symbol/ExternalManager.java
View File

@@ -15,6 +15,7 @@
*/
package ghidra.program.model.symbol;
import java.util.List;
import java.util.Set;
import ghidra.program.model.address.Address;
@@ -31,13 +32,32 @@ import ghidra.util.exception.InvalidInputException;
public interface ExternalManager {
/**
* Returns an array of all external names for which locations have been defined.
* @return array of external names
* Returns an array of all external Library names sorted by preferred search order.
* This order reflects the preferred search order when looking for external symbols
* which were not linked to a specific Library.
* <p>
* NOTE: The {@link Library#UNKNOWN} library will always be returned as first in the list
* but will not have an associated program path and cannot be searched.
*
* @return array of all external Library names sorted by preferred search order.
*/
public String[] getExternalLibraryNames();
/**
* Get a list of all external Libraries sorted by preferred search order.
* This order reflects the preferred search order when looking for external symbols
* which were not linked to a specific Library.
* <p>
* NOTE: The {@link Library#UNKNOWN} library will always be returned as first in the list
* but will not have an associated program path and cannot be searched.
*
* @return list of all external Libraries sorted by preferred search order.
*/
public List<Library> getLibraries();
/**
* Get the Library which corresponds to the specified name
*
* @param libraryName name of library
* @return library or null if not found
*/
@@ -45,6 +65,7 @@ public interface ExternalManager {
/**
* Removes external name if no associated ExternalLocation's exist
*
* @param libraryName external library name
* @return true if removed, false if unable to due to associated locations/references
*/
@@ -54,35 +75,69 @@ public interface ExternalManager {
* Returns the file pathname associated with an external name.
* Null is returned if either the external name does not exist or
* a pathname has not been set.
*
* @param libraryName external name
* @return project file pathname or null
*/
public String getExternalLibraryPath(String libraryName);
/**
* Sets the file pathname associated with an existing external name.
* Sets the file pathname associated with an external name.
* If the Library namespace/symbol does not already exist it will be created provided
* the libraryName does not conflict with another namespace whose parent is the global
* namespace.
* <p>
* NOTE: Assigning path for {@link Library#UNKNOWN} Library will be ignored.
* <p>
* NOTE: Assigning path to a non-Library namespace will fail silently.
*
* @param libraryName the name of the library to associate with a file.
* @param pathname the path to the program to be associated with the library name.
* @param userDefined true if the external path is being specified by the user
* @throws InvalidInputException on invalid input
* @throws InvalidInputException on invalid input specified
*/
public void setExternalPath(String libraryName, String pathname, boolean userDefined)
throws InvalidInputException;
/**
* {@return the ordinal associated with an external library which represents its
* sequence within the order list of libraries or -1 if library name not found}
*
* @param libraryName the library name
*/
public int getLibraryOrdinal(String libraryName);
/**
* Sets the Library search ordinal associated with an external name.
* <p>
* Assigning ordinal for {@link Library#UNKNOWN} Library will fail and return -1.
* Assigning ordinal to a non-existing Library will fail and return -1.
* <p>
* NOTE: The actual ordinal applied my be limited based on placement restrictions.
*
* @param libraryName the name of the library to position within Library search sequence
* @param ordinal library ordinal greater or equal to 1
* @return the actual ordinal applied or -1 if change failed.
*/
public int setLibraryOrdinal(String libraryName, int ordinal);
/**
* Change the name of an existing external name.
*
* @param oldName the old name of the external library name.
* @param newName the new name of the external library name.
* @param source the source of this external library
* @return true if symbol was found and renamed, false if symbol not found
* @throws DuplicateNameException if name conflicts with another symbol.
* @throws InvalidInputException if an invalid or null name specified (see
* {@link SymbolUtilities#validateName}).
*/
public void updateExternalLibraryName(String oldName, String newName, SourceType source)
public boolean updateExternalLibraryName(String oldName, String newName, SourceType source)
throws DuplicateNameException, InvalidInputException;
/**
* Get an iterator over all external locations associated with the specified Library.
*
* @param libraryName the name of the library to get locations for
* @return external location iterator
*/
@@ -91,6 +146,7 @@ public interface ExternalManager {
/**
* Get an iterator over all external locations which have been associated to
* the specified memory address
*
* @param memoryAddress memory address
* @return external location iterator
*/
@@ -151,6 +207,7 @@ public interface ExternalManager {
/**
* Determines if the indicated external library name is being managed (exists).
*
* @param libraryName the external library name
* @return true if the name is defined (whether it has a path or not).
*/
@@ -158,6 +215,7 @@ public interface ExternalManager {
/**
* Adds a new external library name
*
* @param libraryName the new external library name to add.
* @param source the source of this external library
* @return library external {@link Library namespace}
@@ -173,6 +231,7 @@ public interface ExternalManager {
* Get or create an external location associated with a library/file named {@code libraryName}
* and the location within that file identified by {@code extLabel} and/or its memory address
* {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified.
*
* @param libraryName the external library name
* @param extLabel the external label or null
* @param extAddr the external memory address or null
@@ -191,6 +250,7 @@ public interface ExternalManager {
* Create an external location in the indicated external parent namespace
* and identified by {@code extLabel} and/or its memory address
* {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified.
*
* @param extNamespace the external namespace
* @param extLabel the external label or null
* @param extAddr the external memory address or null
@@ -208,6 +268,7 @@ public interface ExternalManager {
* Get or create an external location in the indicated external parent namespace
* and identified by {@code extLabel} and/or its memory address
* {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified.
*
* @param extNamespace the external namespace
* @param extLabel the external label or null
* @param extAddr the external memory address or null
@@ -227,6 +288,7 @@ public interface ExternalManager {
* Create an external {@link Function} in the external {@link Library} namespace
* {@code libararyName} and identified by {@code extLabel} and/or its memory address
* {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified.
*
* @param libraryName the external library name
* @param extLabel label within the external program, may be null if extAddr is not null
* @param extAddr memory address within the external program, may be null
@@ -245,6 +307,7 @@ public interface ExternalManager {
* Create an external {@link Function} in the indicated external parent namespace
* and identified by {@code extLabel} and/or its memory address
* {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified.
*
* @param extNamespace the external namespace
* @param extLabel the external label or null
* @param extAddr the external memory address or null
@@ -262,6 +325,7 @@ public interface ExternalManager {
* Get or create an external {@link Function} in the indicated external parent namespace
* and identified by {@code extLabel} and/or its memory address
* {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified.
*
* @param extNamespace the external namespace
* @param extLabel the external label or null
* @param extAddr the external memory address or null

7
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/util/ChangeManager.java
View File

@@ -357,13 +357,6 @@ public interface ChangeManager {
@Deprecated
public static final ProgramEvent DOCR_MEM_REF_PRIMARY_REMOVED = REFERENCE_PRIMARY_REMOVED;
/**
* The external path name changed for an external program name.
* @deprecated Event type numeric constants have been changed to enums. Use the enum directly.
*/
@Deprecated
public static final ProgramEvent DOCR_EXTERNAL_PATH_CHANGED = EXTERNAL_PATH_CHANGED;
/**
* An external program name was added.
* @deprecated Event type numeric constants have been changed to enums. Use the enum directly.

9
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/util/ProgramEvent.java
View File

@@ -53,9 +53,12 @@ public enum ProgramEvent implements EventType {
SYMBOL_DATA_CHANGED, // some symbol property was changed
SYMBOL_ADDRESS_CHANGED, // the symbol's address changed (only applies to param and variables)
EXTERNAL_ENTRY_ADDED, // an external entry point was added
EXTERNAL_ENTRY_REMOVED, // an external entry point was removed
EXTERNAL_PATH_CHANGED, // the external path name changed for an external program
EXTERNAL_ENTRY_ADDED, // an external entry point was added (i.e., Export)
EXTERNAL_ENTRY_REMOVED, // an external entry point was removed (i.e., Export)
//
// Events related to Library symbols and associated External Location
//
EXTERNAL_NAME_ADDED, // an external program name was added
EXTERNAL_NAME_REMOVED, // an external program name was removed
EXTERNAL_NAME_CHANGED, // the name of an external program was changed

16
Ghidra/Test/IntegrationTest/src/screen/java/help/screenshot/ReferencesPluginScreenShots.java
View File

@@ -38,9 +38,9 @@ import ghidra.program.model.listing.CodeUnit;
import ghidra.program.model.listing.Program;
import ghidra.program.model.mem.Memory;
import ghidra.program.model.mem.MemoryAccessException;
import ghidra.program.model.symbol.ExternalManager;
import ghidra.util.InvalidNameException;
import ghidra.util.exception.CancelledException;
import ghidra.util.exception.VersionException;
import ghidra.util.exception.*;
import ghidra.util.task.TaskMonitor;
public class ReferencesPluginScreenShots extends GhidraScreenShotGenerator {
@@ -122,9 +122,15 @@ public class ReferencesPluginScreenShots extends GhidraScreenShotGenerator {
}
@Test
public void testExternal_names_dialog() {
showProvider(ExternalReferencesProvider.class);
captureProvider(ExternalReferencesProvider.class);
public void testExternal_names_dialog() throws InvalidInputException {
ExternalManager externalManager = program.getExternalManager();
program.withTransaction("Set Program Path", () -> {
externalManager.setExternalPath("USER32.DLL", "/libs/user32.dll", true);
});
ExternalReferencesProvider provider = showProvider(ExternalReferencesProvider.class);
JTable table = findComponent(provider.getComponent(), JTable.class);
selectRow(table, 0);
captureIsolatedProvider(ExternalReferencesProvider.class, 500, 500);
}
@Test