I will try to be more verbose with the release notes this time, it could help everyone to understand better the changes that happen, project directions and new features, as well as be more entertaining, even readable by Siri.
As I'm writing these lines I know I will be missing something, so please, if you think this summary is missing any important details let us know and our apologize in advance!
TLDR
Highlights
This release comes with a large list of bug fixes, many of them you may probably not even noticed, but some of them are important for users and packagers, it has been
tested on a large list of platforms, not just in the CI, but also in sparc, mips, powerpc and other funky hardware (Thanks @unixfreaxjp !). We are not forgetting the new Macs, and
this release comes with few fixes for fat binaries, kernel caches and arm64 floating point emulation (kudos to @mrmacete and @aemmitt-ns for them!).
Projects: One of the most awaited feature is now finally available for testing, the git
support has been enabled by default and some options and backward compatibility transitional
code have been removed. Please test this out and let us know if you spot any issue! thanks @trufae for this!
The CI have been rewritten for simplicity and it's now building and publishing Android, iOS, macOS, Linux and Windows artifacts on every commit, ASAN, LGTM and COVERITY are still there, but all jobs run in github actions.
Lots of improvements in the support for JSON have been added by @liumeo also, several memory leaks have been cutted down, which are always welcome.
Support for streaming large files over mg, and added support in r2frida is now available thanks to @as0ler!
r2wars
The r2wars game runs on top of r2, but it needs some tweaks for the esil vm to work,
this version optimizes this by checking configuration options outside hot loops.
- Cache cfg.r2wars value outside the eval loop
Those 'hacks' will be eventually removed when r2wars gets able to emulate syscalls,
traps and low level stepping for context switching at esil-expression level.
- Support sbfm/ubfm in arm64
- Initial support for arm64 asm extendtype
- Add test and update arm.winedbg (#18117)
The arm64 assembler has been extended support more instructions and be more formal and correct.
- Implement i4004 assembler
It's always great to welcome a new supported architecture for assembling code (disassembler for i4004 was already available). Kudos to Liumeo for this nice addition!
bin
-
COFF: handle empty sections (#447)
-
Dont demangle with libs unless requested
-
Add bin.cache evar to use io.cache when bins need to patch relocs
-
Fix Mach-O rebase on fat slices
-
Add additional ELF header fields to rz-bin output
-
Fix PE Delay Imports for multiple delayed DLLs (rizin)
-
Lowercase DEX method attributes and move r_num_bit_count()
-
Initial implementation of the DEX annotation parser
The DEX annotation metadata is now parsed in the DEX plugin, this means, that parsing is actually a bit slower (it's parsing more information) but provides more context and information of the application classes and methods. Use bin.verbose=true to get that information.
This metadata must be imported into r2 somehow, but this interface hasnt been defined yet, so only plaintext representation is supported at load time. Probably finding a good tree representation for an Sdb instance could work.
radiff2
- Add more checks on the passed files and fail early.
- Honor graph.font in diffing graphs too
- Remove buggy Levenshtein diff algorithm and rename the original code
Some confusing usage and documentation have been updated and the default diffing algorithm is now faster. (Thanks MaskRay for spotting it and Liumeo for massaging it)
ci
The whole CI scripts have been rewritten to run everything in GithubActions, and adjust the test of PRs to 20min, ASAN is only running in master (takes 1h), and every commit is compiled for linux, macos, windows, ios and android. No breaking commits can be merged. And all artifacts are available to download for every single commit and architecture.
As long as the Sanitized build takes 1h to run the testsuite we decided to make it run only in the master branch, if any regression happens there it's easy to fix with the crash logs in GHA.
This is the setup of jobs in the current CI:
- Add android-arm64 target to build release artifacts
- Add TCC ci task which is able to build and run the testuite
- Added cydia builds for arm64
- Add job to test build and install with spaces in builddir and installdir
- Add job to test install, uninstall, symstall for proper purgation and avoid disasters
- Fix the badge in the README
- Add asan ci job to run all fuzzed bins with a sanitized build (takes 1h)
- linux-test builds with acr and takes about 20min to run all tests
- CoverityScan service find vulnerabilities with advanced source code analysis.
- LGTM service spots static source analysis good practices
- Initial attempt to switch to Capstone 5, needs more
RTable
- Dashes in RTable with X format
- Implement RTable:sql and add RTable.name
You may not know about RTable yet, but it's an api and command modifier that will be used more and more over time. In short, RTable provides an API to create tables with columns with types and rows with data and an api and query syntax to operate over those tables in the same way as you would do in an SQL database but using the cryptic syntax of commands we like in r2land.
This release introduces a new output for SQL. This means that any information stored in r2 can be exported in SQL statements and processed in your favourite SQL database. This is an example usage:
$ r2 -AA /bin/ls
> afl,:sql > functions.sql
> !sqlite3
sqlite> .read functions.sql
sqlite> .tables
fcns
sqlite> select count(name) from fcns;
128
RISCV
- Fix #18212 - Detect RISCV gdb servers
- Add riscv in RSysArch and make it an enum, not a bitmask
- Update RISC-V ESIL with sign extention operator (#18109)
Native support for Linux/RISC-V is now available as well as remote debugging via GDB,
the ESIL emulation have been improved a little bit.
disasm
- Fix HUGE bottleneck in the WebAssembly pseudo disassembler and analyzer
- pd, is an alias for pdt (pdt will be removed soon)
- Honor meta size in asm.meta=false and add tests
- Fix #18202 - Large Cd truncates and crashes in pd
- Implement print disasm until optype
ESIL
- Fix x86_cs cmpbs esil
- fix x86-cs rep/repe/repne esil expressions
@condret find out (and fixed) a bug in the way rep instructions were constructed in ESIL in x86.
- Support arm32 esil stmib/ldmib
- Add sign extension assignment operator (#18092)
- Add floating point operations for emulation
Thanks to @aemmitt-ns (Austin Emmitt) for implementing support for floating point arithmetics in ESIL as well as adding support for most FPU instructions for ARM64. That's an important move forward in order to improve the language to handle more instructions and architectures.
A work in progress support for RIOBanks is not yet included in this release.
But hopefully in the next release @condret and @trufae will be manage to finish the new API and commands and integrate them into the ESIL to support memory banks in GameBoy emulation for example (as well as add support later for other archs).
- Add support for RAnal.ESIL plugins
Those new types of plugins are right now just a place holder to call init/fini and do whatever you want from there. But in the next release ESIL plugins will provide the ability to expose some functionalities to the ESIL VM, syscall implementations in userland, libc emulation functions, custom esil operations, hardware devices, etc. Join the Discord, Telegram or IRC channels to raise the topic if you are interested on more details.
fs
- Add support to stream files using mg (#18253)
This feature has been added pair to pair with the r2frida implementation, this
way enables r2 to download all the files and its contents without any file size
limitation from the remote device to your host. All the RFS plugins API has been
changed to if you are using custom RFS plugins you may take care of that.
Support for uploading is not yet implemented, but it is planned in the near future.
Thanks Murphy for that great contrib!
io
- Accept rwx argument in onn command
- Add onn command to fix custom map assignments
Those changes and new command are required for the projects to be able to save and
restore the status of files, binfile and iomaps in proper order and reference.
- Fix and refactor the ar:// plugin
The refactoring of the io.ar plugin spotted a regression in open_many() which is not yet
fixed, brave volunteers are welcome!
Projects
The most requested feature for r2 has been reworked to actually
make it work and improved several use cases that weren't handled
before:
- prj.git is now enabled by default if
git is in path
this means that everytime you run Ps after saving the changes
in will prompt you for a commit message. The ability to rollback
to any previous state of the project by just calling git reset
and Po becomes very handy when bad things happen or you just
want to track your progress.
As long as projects are in plain text they are readable in git diff.
Improved support to ease the workflow to support multiple users sharing
the same project via git will be implemented in future releases.
- Handle io.maps and bin.segments in o* to handle custom maps in projects
- Add map name information in
o* output
- Honor mapaddr for malloc in o*
- Save the write cache in projects
As long as the user can create custom maps on specific files, the projects
need to determine if there's any binobject associated with a specific
file for processing a map. This puzzle is solved by the o* command which
now prints the right commands to reconstruct the same IO environment starting
from a clean session.
- Reworked P command with RProject and prj.name integration
- dir.projects becomes abspath when set
- Fix projects by removing code and honoring prj.name
- Save the write cache in projects
The P command is now much more stable and all the subcommands work as expected, some tests have been added and project renaming can be done via command or via evar prj.name. The magic behind this evar-project-action is done by using the RConfig.getter APIs that have been there for a while but barely used, the value is updated at get time from the project instance details. This way it's possible to rename a project like this:
> e prj.name
test
> e prj.name=case1
- Tell the user that debugging projects don't work
Projects are working, but they are far from perfect, one of these missing corners is the
debugging support, the main reason for that is the lack of integration of aslr rebasing in projects, this will be eventually implemented, but for now it's better to avoid the user to mess the thing.
In any case, it's always recommended to have your own manually writen scripts to do setup some flags, memory patches or breakpoints, so you are more in control of what you run in a living process.
- Fix calling convention save/restore
- Print call convention once in afi
- Warn once about the missing anal.cc
- Use RConfigNode.getter callback in anal.cc to be in sync with
k anal/cc/default.cc
Some improvements in the way calling conventions are handled inside r2 enabled the use of anal.cc like it's done in prj.name, with a 'live' evar. default calling convention is
defined by the architecture but can be redefined by the rbin plugin or the analysis information. In addition the user can also specify a custom CC for each function, all those details are preserved with the anal.cc evar and the tc and afc commands.
- Remove transitional projects code
- Remove file.path and file.lastpath and add RProject
- Remove the prj.simple option
- dir.projects becomes abspath when set
- Use UID instead of PID to identify the user to avoid changing projects everytime
Refactor
- Refactor tcc and afcl commands, improve help and JSON
- 25 commits refactoring the code to use the formal PJ api to generate JSON
This includes honoring the settings defined by the user in the cfg.json evars,
this nice feature was introduced by @hexploitable in the previous 5.0 release.
[0x00000000]> e cfg.json.num =?
none
string
hex
Rizin
- Added support for regex in test output and stderr
- Massage RRegex to fix codingstyle and a null deref.
- This resulted in a cleanup and refactoring of RRegex
One of the changes introduced in Rizin is the ability to use regexps to check the output of an r2r test, but after doing some cleanup in the regex code some issues were spotted in the logic, so it's not encouraged to write tests using regexps yet. Unit tests has been added, but it still requires to be fixed.
The bugs are logic bugs, not exploitable, but some match expressions won't work. But at least the feature is in sync.
Other commits taken from RZ grouped by author are:
Paul I
- COFF empty sections
- memleaks in ophandlers
- rtable X dashes
xvilka
- part of the improvements for indentation
wargio
- avr anal warning due unpopulated mnemonic and further refactoring
- fix ao rjmp issue
ret2libc
- Fix misusess of r2 commands inside r2
- Use r_core_flag_get_by_spaces() in getFunctionName()
kazarmi
- Fixed AVR anal plugin warning due unpopulated mnemonic
- Fixing clang flow warnings (#321)
- Fix #rizin302 - Fix function modification detection false
- Remove all dead assignment detect by clang sa (#310)
yossizap
- Fix trace crash
- Add regex support in r2r
Florian
- Fix null deref in rbtree
- Implement delay imports in PE parser
shell
- Implement rax2 -I to convert from/to LONG and IP Address
Sometimes a shellcode or a piece of program is doing some operations with IP addresses and it stores the IP address on a 32 bit register value. rax2 now provides a handy commandline option to ease this conversion. This feature was already available as a hint for the disassembly to convert instruction arguments to ip addresses.
$ rax2 -I 192.168.1.32
0x2001a8c0
$ rax2 -I 0x2001a8c0
192.168.1.32
$
- Implement $i and $I numvars
Those two variables have been added in order to ease writing some scripts that navigate thru
the code moving forward and backward honoring the instruction boundaries of the current analysis information.
So $i is the address of the next instruction and $I of the previous. Things get more interesting when the braces join the game: Using $i{3} gives you the address of 3 instructions forward. and the same goes for $I{3} to go backward.
- Fix #18171 - Support RNum for syscall-name in
asl command
The asl command has been modified to use RNum when parsing the argument, so its possible to
API
- Add RFile.new and RFile.move APIs
- RFSPlugin API has changed
Plugin delegates return int instead of RFSFile to avoid leaks and uafs, needed for streaming largs files over mg.
Add esil.dummy in your plugins.cfg if the build fail with missing R_ESIL_PLUGINS error.
Visual
- Fix ecn (and VR) when no custom theme was set in .rc
In human words: rotating color themes is working again!
- Implement history filtering for dietline
- Initial implementation of r_cons_eprintf
This new API is wrapping eprintf() but its also able to buffer the
results and flush them after r_cons_flush().
FUTURE: The need for this API is to improve the r2pipe API and handle
a 3rd communication pipe to handle asyncronous error messages. This is
a long term plan and should be backward compatible, so no r2pipe scripts may break.
- Fix help rendering: avoid printing trailing whitespaces (#18115)
- Improve str.wrap, add cons.line and fix cons.printat glitch
- Fix #17940 - Show ConfigNode options when selected in Ve
- Box borders in graph and panels are now in yellow
- Update www/t from radare2-webui
- Fix cascading solitaire issue in panels menus
- Fix the 'c' cursor behaviour in disasm
Those commits improve the experience in panels, fixing an anoying
bug in the menus, improving the cursor mode. The default color
theme for the frames makes it easier the eye.
The heavy webuis were removed in 5.0, but we are still shipping
t/iled and p/anel ones, it's known that the webui repo needs some
attention
bindings
The bindings have been also updated with some more valadoc documentation
that can be read in here:
https://radare.org/vdoc
This documentation and API can be used for any bindings generated by
valabind, this is: python, nodejs, ruby, go, v, ... the work to stabilize
the apis in this module focused in RConfig, some fixes have been done in
this module.
security
As usual, every release of r2 comes with a large list of security vulnerabilities, bugs and crash fixes. The list below sumarizes the most relevant ones:
- Fix #18274 - Fix crash in r2 *.wasm
- Fix crash in XNU kernel parsing (no cache)
- Fix code injection vuln in .ic* with ObjC classes
- Fix trace crash caused by a mismatch between the register profile and op anal
- AVR: Fixed profile, (null) instruction and anal
- Fix potential null-deref in r_rbtree_cont_foreach()
- Fix crash when wasm file contains symbols with large names
- Handle ^C and fix ASAN crash in aeA command
build
- Disable AVR plugin from all static builds because of the duplicated symbols issue introduced in recent refactoring.
- Add r2.1 when installing with
symstall
- Fix debugger build problem in android-x86_64
- Remove --without-r2r configure option
- Create dist/ to hold all the distribution build files
Merged some patches coming from Termux to improve the debugger support on android-x86. The r2r testsuite executable is always built and should be available to all the user installations.
Also, some issues has been fixed in sys/install as well as new CI jobs to verify no regressions happen on install/uninstall/spacesinpaths, etc.
config
- Fix some returns to fix initialization issues in evars
- Remove unused cmd.xterm and use * instead of strcmp for ?
- Support evar filtering in eq and check for bool type in RConfig.toggle
- Expose RConfigNode.options APIs to avoid messing with internals
- Count lines is a prefix operator
- Add a progress bar for when scripts are running
- Honor R2_CFG_NEWSHELL=0 to disable it
- Seek command ignores the tmpseek
- Add missing vars from ?$? in ?$ and sort them alphabetically
1.5k Dec 23, 2022
3 Feb 3, 2022
1.6k Dec 29, 2022
2.1k Jan 3, 2023
13.2k Jan 4, 2023
2.5k Jan 2, 2023
5 Sep 3, 2022
418 Dec 29, 2022
4.1k Dec 25, 2022
2k Dec 26, 2022
4.3k Jan 2, 2023
158 Oct 18, 2022
636 Dec 18, 2022
2 Aug 29, 2021
5 Oct 18, 2022
92 Dec 15, 2022
139 Dec 12, 2022
231 Dec 29, 2022
173 Jan 2, 2023