dbSta: Crash in STA callback during repair_tie_fanout

[jhkim-pii]

Describe the bug

repair_tie_fanout command caused a segmentation fault in a flat flow.

Design: gf12/bp_dual

Repair tie lo fanout...
Signal 11 received
Stack trace:
0# 0x00005D943EDFE3DD in openroad
1# 0x00007C80F2445330 in /lib/x86_64-linux-gnu/libc.so.6
2# sta::Sdc::disconnectPinBefore(sta::Pin const*) in openroad
3# sta::Sta::disconnectPinBefore(sta::Pin const*) in openroad
4# sta::dbStaCbk::inDbITermPreDisconnect(odb::dbITerm*) in openroad
5# odb::dbITerm::disconnect() in openroad
6# rsz::Resizer::createNewTieCellForLoadPin(sta::Pin const*, char const*, sta::Instance*, sta::LibertyPort*, int) in openroad
7# rsz::Resizer::repairTieFanout(sta::LibertyPort*, double, bool) in openroad
8# rsz::repair_tie_fanout_cmd(sta::LibertyPort*, double, bool) in openroad
9# 0x00005D943F000B01 in openroad
10# TclNRRunCallbacks in /lib/x86_64-linux-gnu/libtcl8.6.so
11# 0x00007C80F74396BC in /lib/x86_64-linux-gnu/libtcl8.6.so
12# Tcl_EvalEx in /lib/x86_64-linux-gnu/libtcl8.6.so
13# Tcl_Eval in /lib/x86_64-linux-gnu/libtcl8.6.so
14# sta::sourceTclFile(char const*, bool, bool, Tcl_Interp*) in openroad
15# 0x00005D943EDFF508 in openroad
16# ord::tclInit(Tcl_Interp*) in openroad
17# ord::tclAppInit(Tcl_Interp*) in openroad
18# Tcl_MainEx in /lib/x86_64-linux-gnu/libtcl8.6.so
19# main in openroad
20# 0x00007C80F242A1CA in /lib/x86_64-linux-gnu/libc.so.6
21# __libc_start_main in /lib/x86_64-linux-gnu/libc.so.6
22# _start in openroad

Expected Behavior

No segmentation fault.

Environment

-- The CXX compiler identification is GNU 13.3.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- OpenROAD version: v2.0-26058-g842273967c
-- System name: Linux
-- Compiler: GNU 13.3.0
-- Build type: DEBUG
-- Install prefix: /usr/local
-- C++ Standard: 17
-- C++ Standard Required: ON
-- C++ Extensions: OFF
-- The C compiler identification is GNU 13.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test C_COMPILER_SUPPORTS__-Wall
-- Performing Test C_COMPILER_SUPPORTS__-Wall - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wall
-- Performing Test CXX_COMPILER_SUPPORTS__-Wall - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-array-bounds
-- Performing Test C_COMPILER_SUPPORTS__-Wno-array-bounds - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-array-bounds
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-array-bounds - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-nonnull
-- Performing Test C_COMPILER_SUPPORTS__-Wno-nonnull - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-nonnull
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-nonnull - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-maybe-uninitialized
-- Performing Test C_COMPILER_SUPPORTS__-Wno-maybe-uninitialized - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-maybe-uninitialized
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-maybe-uninitialized - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format-overflow
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format-overflow - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format-overflow
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format-overflow - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-variable
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-variable - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-variable
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-variable - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-function
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-function - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-function
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-function - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-write-strings
-- Performing Test C_COMPILER_SUPPORTS__-Wno-write-strings - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-write-strings
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-write-strings - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-sign-compare
-- Performing Test C_COMPILER_SUPPORTS__-Wno-sign-compare - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-sign-compare
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-sign-compare - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-deprecated
-- Performing Test C_COMPILER_SUPPORTS__-Wno-deprecated - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-deprecated
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-deprecated - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-c++11-narrowing
-- Performing Test C_COMPILER_SUPPORTS__-Wno-c++11-narrowing - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-c++11-narrowing
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-c++11-narrowing - Failed
-- Performing Test C_COMPILER_SUPPORTS__-Wno-register
-- Performing Test C_COMPILER_SUPPORTS__-Wno-register - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-register
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-register - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal
-- Performing Test C_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal - Failed
-- Performing Test C_COMPILER_SUPPORTS__-fpermissive
-- Performing Test C_COMPILER_SUPPORTS__-fpermissive - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-fpermissive
-- Performing Test CXX_COMPILER_SUPPORTS__-fpermissive - Success
-- Performing Test C_COMPILER_SUPPORTS__-x
-- Performing Test C_COMPILER_SUPPORTS__-x - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-x
-- Performing Test CXX_COMPILER_SUPPORTS__-x - Failed
-- Performing Test C_COMPILER_SUPPORTS__c++
-- Performing Test C_COMPILER_SUPPORTS__c++ - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__c++
-- Performing Test CXX_COMPILER_SUPPORTS__c++ - Failed
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-but-set-variable
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-but-set-variable - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-but-set-variable
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-but-set-variable - Success
-- Performing Test C_COMPILER_SUPPORTS__-std=c++17
-- Performing Test C_COMPILER_SUPPORTS__-std=c++17 - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-std=c++17
-- Performing Test CXX_COMPILER_SUPPORTS__-std=c++17 - Success
-- Performing Test C_COMPILER_SUPPORTS__-fno-exceptions
-- Performing Test C_COMPILER_SUPPORTS__-fno-exceptions - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-fno-exceptions
-- Performing Test CXX_COMPILER_SUPPORTS__-fno-exceptions - Success
-- Found Python3: /usr/bin/python3.12 (found version "3.12.3") found components: Interpreter
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- TCL library: /usr/lib/x86_64-linux-gnu/libtcl.so
-- TCL header: /usr/include/tcl/tcl.h
-- TCL readline library: /usr/lib/x86_64-linux-gnu/libtclreadline.so
-- TCL readline header: /usr/include/x86_64-linux-gnu
-- Found SWIG: /usr/local/bin/swig (found suitable version "4.3.0", minimum required is "4.0")
-- Using SWIG >= 4.3.0 -flatstaticmethod flag for python
-- boost: 1.86.0
-- Found GTest: /usr/local/lib/cmake/GTest/GTestConfig.cmake (found version "1.13.0")
-- GTest: 1.13.0
-- Found Python3: /usr/include/python3.12 (found version "3.12.3") found components: Development Development.Module Development.Embed
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.3")
-- spdlog: 1.15.0
-- Found BISON: /usr/bin/bison (found version "3.8.2")
-- Found Doxygen: /usr/bin/doxygen (found version "1.9.8") found components: doxygen dot
-- STA version: 2.7.0
-- STA git sha: 19202ac93331f547fa16f7c4697e573cb513842b
-- System name: Linux
-- Compiler: GNU 13.3.0
-- Build type: DEBUG
-- Build CXX_FLAGS: -g
-- Install prefix: /usr/local
-- Found FLEX: /usr/bin/flex (found version "2.6.4")
-- TCL library: /usr/lib/x86_64-linux-gnu/libtcl.so
-- TCL header: /usr/include/tcl/tcl.h
-- TCL readline library: /usr/lib/x86_64-linux-gnu/libtclreadline.so
-- TCL readline header: /usr/include/x86_64-linux-gnu/tclreadline.h
-- CUDD library: /usr/local/lib/libcudd.a
-- CUDD header: /usr/local/include/cudd.h
-- SSTA: 0
-- Found SWIG: /usr/local/bin/swig (found suitable version "4.3.0", minimum required is "3.0")
-- STA library: /workspace/ws5/OpenROAD-flow-scripts/tools/OpenROAD/build/libOpenSTA.a
-- STA executable: /workspace/ws5/OpenROAD-flow-scripts/tools/OpenROAD/build/sta
-- Found re2: /opt/or-tools/lib/cmake/re2/re2Config.cmake (found version "11.0.0")
-- Found Clp: /opt/or-tools/lib/cmake/Clp/ClpConfig.cmake (found version "1.17.7")
-- Found Cbc: /opt/or-tools/lib/cmake/Cbc/CbcConfig.cmake (found version "2.10.7")
-- Found SCIP: /opt/or-tools/lib/cmake/scip/scip-config.cmake (found version "9.0.0")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Found OR-Tools: /opt/or-tools/lib/cmake/ortools (version: 9.11.4210)
-- TCL library: /usr/lib/x86_64-linux-gnu/libtcl.so
-- TCL header: /usr/include/tcl/tcl.h
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- GUI is enabled
-- Could NOT find VTune (missing: VTune_LIBRARIES VTune_INCLUDE_DIRS) 
-- TCL library: /usr/lib/x86_64-linux-gnu/libtcl.so
-- TCL header: /usr/include/tcl/tcl.h
-- Found Eigen3: /usr/local/share/eigen3/cmake/Eigen3Config.cmake (found version "3.4.1")
-- TCL readline enabled
-- Tcl Extended disabled
-- Python3 enabled
-- Configuring done (4.2s)
-- Generating done (0.6s)

To Reproduce

Cannot share the design

Relevant log output

Screenshots

No response

Additional Context

GDB stack trace

sta::Sdc::disconnectPinBefore(sta::Sdc * const this, const sta::Pin * pin) (\workspace\ws5\OpenROAD-flow-scripts\tools\OpenROAD\src\sta\sdc\Sdc.cc:5632)
sta::Sta::disconnectPinBefore(sta::Sta * const this, const sta::Pin * pin) (\workspace\ws5\OpenROAD-flow-scripts\tools\OpenROAD\src\sta\search\Sta.cc:4505)
sta::dbStaCbk::inDbITermPreDisconnect(sta::dbStaCbk * const this, odb::dbITerm * iterm) (\workspace\ws5\OpenROAD-flow-scripts\tools\OpenROAD\src\dbSta\src\dbSta.cc:1003)
odb::dbITerm::disconnect(odb::dbITerm * const this) (\workspace\ws5\OpenROAD-flow-scripts\tools\OpenROAD\src\odb\src\db\dbITerm.cpp:522)
rsz::Resizer::createNewTieCellForLoadPin(rsz::Resizer * const this, const sta::Pin * load_pin, const char * new_inst_name, sta::Instance * parent, sta::LibertyPort * tie_port, int separation_dbu) (\workspace\ws5\OpenROAD-flow-scripts\tools\OpenROAD\src\rsz\src\Resizer.cc:3446)
rsz::Resizer::repairTieFanout(rsz::Resizer * const this, sta::LibertyPort * tie_port, double separation, bool verbose) (\workspace\ws5\OpenROAD-flow-scripts\tools\OpenROAD\src\rsz\src\Resizer.cc:3338)
rsz::repair_tie_fanout_cmd(sta::LibertyPort * tie_port, double separation, bool verbose) (\workspace\ws5\OpenROAD-flow-scripts\tools\OpenROAD\build\src\rsz\src\CMakeFiles\rsz.dir\ResizerTCL_wrap.cxx:2340)
_wrap_repair_tie_fanout_cmd(ClientData clientData, Tcl_Interp * interp, int objc, Tcl_Obj * const * objv) (\workspace\ws5\OpenROAD-flow-scripts\tools\OpenROAD\build\src\rsz\src\CMakeFiles\rsz.dir\ResizerTCL_wrap.cxx:3875)
libtcl8.6.so!TclNRRunCallbacks (Unknown Source:0)
libtcl8.6.so![Unknown/Just-In-Time compiled code] (Unknown Source:0)
libtcl8.6.so!Tcl_EvalEx (Unknown Source:0)
libtcl8.6.so!Tcl_Eval (Unknown Source:0)
sta::sourceTclFile(const char * filename, bool echo, bool verbose, Tcl_Interp * interp) (\workspace\ws5\OpenROAD-flow-scripts\tools\OpenROAD\src\sta\app\StaMain.cc:105)
tclAppInit(int & argc, char ** argv, const char * init_filename, Tcl_Interp * interp) (\workspace\ws5\OpenROAD-flow-scripts\tools\OpenROAD\src\Main.cc:494)
ord::tclInit(Tcl_Interp * interp) (\workspace\ws5\OpenROAD-flow-scripts\tools\OpenROAD\src\Main.cc:538)
ord::tclAppInit(Tcl_Interp * interp) (\workspace\ws5\OpenROAD-flow-scripts\tools\OpenROAD\src\Main.cc:533)
libtcl8.6.so!Tcl_MainEx (Unknown Source:0)
main(int argc, char ** argv) (\workspace\ws5\OpenROAD-flow-scripts\tools\OpenROAD\src\Main.cc:307)

It looks like from is a stale pointer.

[povik]

@AcKoucher this looks similar to your crash

[jhkim-pii]

This commit is the culprit parallaxsw/OpenSTA@5b312ee.
No crash if OpenSTA version is rolled back to the commit before 5b312ee.

[jhkim-pii]

Just reverting the 5b312ee commit by git revert 5b312ee is sufficient to avoid the crash.

[AcKoucher]

As @povik pointed out, I'm seeing the same crash in my Secure-CI run for #8375.

[maliberty]

I just updated sta to get a fix for @arthurjolo . Please see if it still reproduces with the head of OR.

[AcKoucher]

I updated my branch with OR master and I still get the crash.

[jhkim-pii]

I think I found the cause.

Cause

• In parallaxsw/OpenSTA@5b312ee, pin_exceptions_ map is newly introduced.
• Sdc::recordException(ExceptionPath*) inserts element in pin_exceptions_ map.
• But Sdc::unrecordException(ExceptionPath*) does not remove the inserted element in pin_exceptions_ map, which made stale pointers.

tools/OpenROAD/src/sta/sdc/Sdc.cc

void
Sdc::recordException(ExceptionPath *exception)
{
  exceptions_.insert(exception);
  exception->setId(++exception_id_);
  recordMergeHashes(exception);
  recordExceptionFirstPts(exception);
  recordExceptionPins(exception);            // INSERT INTO pin_exceptions_ map
  checkForThruHpins(exception);
}

void
Sdc::unrecordException(ExceptionPath *exception)
{
  unrecordMergeHashes(exception);
  unrecordExceptionFirstPts(exception);
  exceptions_.erase(exception);
                                           // unrecordExceptionPins(exception) IS MISSING
}

Solution

Implement and call unrecordExceptionpins(ExceptionPath*).
No crash with this code change.

void
Sdc::unrecordException(ExceptionPath *exception)
{
  unrecordMergeHashes(exception);
  unrecordExceptionFirstPts(exception);
  unrecordExceptionPins(exception);       // FIX
  exceptions_.erase(exception);
}

void
Sdc::unrecordExceptionPins(ExceptionPath *exception)           // NEW API
{
  ExceptionFrom *from = exception->from();
  if (from)
    unrecordExceptionPins(exception, from->pins(), pin_exceptions_);
  ExceptionThruSeq *thrus = exception->thrus();
  if (thrus) {
    for (ExceptionThru *thru : *thrus)
      unrecordExceptionPins(exception, thru->pins(), pin_exceptions_);
  }
  ExceptionTo *to = exception->to();
  if (to)
    unrecordExceptionPins(exception, to->pins(), pin_exceptions_);
}

[jhkim-pii]

Filed an OpenSTA issue parallaxsw/OpenSTA#327 with a test case.

[jhkim-pii]

Resolved by OpenSTA commit parallaxsw/OpenSTA@4dad6ab