My HowTos for Android

Last Update: 08.10.2024/bs

My HowTos for Android

Introduction

On this page I mirror my HowTos regarding Android which I published elsewhere (for example in the XDA Forums). The page also contains some HowTos I did not (yet) publish elsewhere.

The list of my HowTos in the XDA Forums can be found using these searches:

https://xdaforums.com/f/asus-zenfone-8.12291/?prefix_id=37&starter_id=8498037

https://xdaforums.com/search/71436794/?q=%5BGUIDE%5D&t=post&c[title_only]=1&c[users]=bnsmb&o=relevance

There is also a section with links to other useful URLs for using or developing Android at the end of this page.

Update 29.07.2024

There is now a copy of this HTML page with all details sections open: My Howtos for Android with open details. This page can be used for searching within the HTML page, for example.
I'm trying to keep this page in sync with the original page, but I can't promise that.

Use this link to get back to the page with closed details section.

Update 13.08.2024

The XDA forum admins moved some of my posts to other sections. This is not a big issue because the links to the posts in this page and elsewhere still work.

Update 15.08.2024

I have thoroughly cleaned up this HTML page. If you find an error in any of the sections here, please contact me: Bernd dot schemmer at gmx dot de.

Back to my home page

History

ChangeLog

08.10.2024

added the section Documentation for the Magisk Module with Python 3.14

05.10.2024

added the section Magisk Module documentation
added the section Documentation for the Magisk Module with OpenSSSH
added the section Documentation for the Magisk Module with the network tools

02.10.2024

updated the section Compiling C programs in Android using gcc (see also here)

30.09.2024

added the section Compiling C programs in Android using gcc

22.09.2024

added the section How to install Magisk modules via a script

11.09.2024

added the section How to compile the OrangeFox Recovery for the ASUS Zenfone 8

09.09.2024

updated and published the section How to create logical devices for disk image files

08.09.2024

added the section How to create logical devices for disk image files

07.09.2024

added the section How to create a new dynamic partition in the partition super
added the section How to access the system partitions while the phone is booted into the LineageOS recovery
updated the section How to create logical devices using the device mapper from Android (see also here)

06.09.2024

added the section How to create logical devices using the device mapper from Android

29.08.2024

published the section Hints for writing init scripts for Magisk
added the section How to add missing SELinux policies dynamically using Magisk

28.08.2024

added the section How to use a log host within the Android operating system
updated the section Some hints for SELinux usage in Android (see also here)
added the section Some hints for creating Magisk Modules
updated the section How to re-sign an apk file (corrected at typo: $ZIPALIGN} instead of ${ZIPALIGN} and added the infos about the script to resign an apk file)
updated the section How to add missing SELinux policies dynamically using Magisk

25.08.2024

added infos about new Magisk modules
added the section Hints for writing init scripts for Magisk
added the instructions to compile NcFTP using the Android SDK to the Examples for compiling C programs for Android using gcc or clang (see also here)

19.08.2024

updated the troubleshooting hints in the section with the Examples for compiling C programs for Android using gcc or clang (see also here)

18.08.2024

added some more info to the instructions to compile Perl 5.40 using the Android SDK to the Examples for compiling C programs for Android using gcc or clang (see also here)
updated the section How to start or stop an app via CLI command (see also here)

17.08.2024

added the section How to start or stop an app via CLI command

16.08.2024

corrected some typos in the code in various sections introduced while doing the page cleanup
added another error and solutiion to the section with trouble shooting hints for compiling programs for Android
added instructions to compile Perl 5.40 using the Android SDK to the Examples for compiling C programs for Android using gcc or clang (see also here)

13.08.2024

added instructions to compile a static linked nmon executable to the Examples for compiling C programs for Android using gcc or clang (see also here)

12.08.2024

added instructions to compile vim to the Examples for compiling C programs for Android using gcc or clang
added instructions to compile a static linked bvi executable to the Examples for compiling C programs for Android using gcc or clang (see also here)
created a new TWRP image for the ASUS Zenfone 8 -- see this list

11.08.2024

added the section How to apply an Android patchlevel to the repositories for the StatiXOS

10.08.2024

added another TWRP image (see here) and added code to the script install_os_via_twrp.sh to handle OmniROM OS updates with patch level 8/2024

09.08.2024

added the section How to compile Perl for Android

07.08.2024

added some Magisk Modules with standard Unix programs compiled for Android

05.08.2024

added instructions to compile OpenSSL for arm64 to the section Examples for compiling C programs for Android using gcc or clang (see also here)

04.08.2024

updated the section How to install packages (apk files) for Android via script

01.08.2024

published the section How to list bind mounts

28.07.2024

updated the list with Magisk modules
updated the section How to list bind mounts
published the section Compiling C programs in Android
published the section Some hints for SELinux usage in Android

27.07.2024

added the section How to list bind mounts
updated the section Compiling C programs in Android (see also here)

26.07.2024

updated the list with Magisk modules

25.07.2024

added the section Compiling C programs in Android

24.07.2024

updated the list with Magisk modules

23.07.2024

added the section Some hints for SELinux usage in Android
updated the section How to automatically start adbd as root user after starting the phone (see also here)
added Magisk module with SELinux tools for Android

22.07.2024

moved the list of public available Magisk modules to a separate table
added the section Hints for writing init scripts for Magisk

21.07.2024

added the section How to automatically start adbd as root user after starting the phone

18.07.2024

updated the section How to install Magisk into the boot partition using a script (the links did not work -- see also here)
updated the section How to create a TWRP image for LineageOS based ROMs using a script (the download link did not work -- see also here)

14.07.2024

updated the section How to apply the Android Security patches to a local repository (see also here)
updated the section How to add the ASUS Hardware Testtool as prebuild app for the StatiXOS ROM (see also here)
updated the section How to boot a phone from a TWRP recovery image via script (see also here)
updated the section How to install Magisk into the boot partition using a script (see also here)
updated the section How to backup the data from the phone using rsync and ssh (including some hints for using sshd on an Android phone)
updated the section How to fix the error Cannot load Android system. Your data may be corrupt.
updated the section How to enable adb via WiFi

08.07.2024

fixed some typos; added infos for some of the links

04.07.2024

updated the section Some hints about adb and fastboot usage in Linux (see also here)

01.07.2024

updated the section How to compile the OmniROM 14 for the the ASUS Zenfone 8 (see here)

29.06.2024

updated the section How to configure the WiFI in Android via CLI command
updated the section How to backup the data from the phone using rsync and ssh (including some hints for using sshd on an Android phone)

27.06.2024

added the section How to create a TWRP image for LineageOS based ROMs using a script

25.06.2024

updated the section How to use TWRP if LineageOS 20.x is installed
updated and published the section How to add files to the ramdisk used for the recovery boot

24.06.2024

added the section How to install "fastboot images"?
updated the section Some hints for fixing errors building an Custom ROM Android image (see also here)

23.06.2024

added the section How to change files in the boot image using Magisk if there is no ramdisk in the boot partition
fixed some typos in the section How to run a script at shutdown
added the section How to add files to the ramdisk used for the recovery boot

11.06.2024

updated the section Some hints about EDL mode

26.05.2024

updated and published the section Some hints about adb and fastboot usage in Linux (see also here)
updated the section How to install an OS image via sideload using the TWRP binary twrp (see here)
added the section How to add the ASUS Hardware Testtool as prebuild app for the StatiXOS ROM
updated the section How to add additional prebuild apks to a custom ROM (see here)

25.05.2024

updated and published the section Some hints about adb and fastboot usage in Linux

24.05.2024

updated and published the section Some hints about adb and fastboot usage in Linux (see also here)

22.05.2024

updated the section How to install and configure the Android OS (see also here)

21.05.2024

published the section How to reset the USB port used to connect an Android phone

20.05.2024

added the section How to boot the phone into the bootloader using the physical buttons
added the section How to reset the USB port used to connect an Android phone
published the section Some hints about EDL mode

19.05.2024

updated the section How to "simulate" prebuild apps using Magisk (see also here)

16.05.2024

added the section Some hints about EDL mode

15.05.2024

published the section How to add additional prebuild apks to a custom ROM
published the section How to "simulate" prebuild apps using Magisk
published the section How to add the ASUS Hardware Testtool as prebuild app for a Custom ROM
updated the section ~~Some hints about the adb daemon in Linux~~ Some hints about adb and fastboot usage in Linux

14.05.2024

added the section How to add missing SELinux policies dynamically using Magisk
added the section How to "simulate" prebuild apps using Magisk
updated the section Instructions to use the ASUS Hardware Testtool
published the section How to re-sign an apk file
added the section How to add the ASUS Hardware Testtool as prebuild app for a Custom ROM

11.05.2024

added the section How to add additional prebuild apks to a custom ROM
added the section How to re-sign an apk file

07.05.2024

updated the section How to install and configure other ROMs via script

03.05.2024

added the section How to install and configure other ROMs via script
updated the section How to install and configure the Android OS (see also here)
update the section How to install packages (apk files) for Android via script
added the section ~~Some hints about the adb daemon in Linux~~ Some hints about adb and fastboot usage in Linux

28.04.2024

added the section How to install an OS image via sideload using the TWRP binary twrp

26.04.2024

Removed the TWRP image for use with LineageOS 20.x
updated the section How to use TWRP if LineageOS 20.x is installed
updated the section How to install and configure the Android OS (see also here)

21.04.2024

cleanup for the section with the links (e.g. delete duplicate links, delete now dead links, etc)

14.04.2024

added a Magisk Module with Android SDK binaries compiled for the aarch64 CPU -- see the list with Magisk Modules below

13.04.2024

added the table with the Magisk Modules
updated the Magisk Module for btop

12.04.2024

added the section How to fix a hanging restore in Titanium Backup

09.04.2024

added the section How to print the security patch level used in repositories or ROM image zip files

08.04.2024

added the section How to sign a zip file with a Custom ROM image
added the section How to get the security patch level in a ROM image zip file

04.04.2024

updated the section How to apply the Android Security patches to a local repository (see also here)
moved the less important infos about applying an Android Security Patch level to a local repository tree to the separate section Observations while checking how to apply the Android Security patches to a local repository

01.04.2024

updated the Tool chain HowTos

31.03.2024

updated and published the section Infos for building a CustomROM image

29.03.2024

added the section Infos for building a CustomROM image
updated the section Some hints for fixing errors building an Custom ROM Android image (see also here)

18.03.2024

added more details in the section How to change any file or directory using Magisk

10.03.2024

updated the section Examples for compiling C programs for Android using gcc or clang (see also here)

02.03.2024

updated the section Some hints for fixing errors building an Custom ROM Android image (see also here)

19.02.2024

corrected some links in section with links
added missing infos to some table entries in the section with links
updated the section Examples for compiling C programs for Android using gcc or clang (see also here)

18.02.2024

updated the section Examples for compiling C programs for Android using gcc or clang (see also here)

17.02.2024

added the section Examples for compiling C programs for Android using gcc or clang

12.02.2024

added the section How to compile a C program for Android
added the section How to compile a C program for Android using gcc

09.02.2024

updated the section Some hints for fixing errors building an Custom ROM Android image (see also here)
added the table with useful Android apps

08.02.2024

updated the section How to compile the OmniROM 14 for the the ASUS Zenfone 8 (see here)
added the section Some hints for fixing errors building an Custom ROM Android image
updated and published the section Some hints for repo sync troubleshooting (see also here)
updated the section Some hints about the Android safe mode (see also here)

07.02.2024

updated the section How to apply the Android Security patches to a local repository (see also here)

05.02.2024

added the section How to get the application name and version from an apk file
updated the section How to compile Magisk
updated the section Some hints for repo sync troubleshooting (see also here)

04.02.2024

added the section How to compile Magisk
added the section repo sync troubleshooting

29.01.2024

Updated the section How to enable and configure the Magisk DenyList via cli commands

22.01.2024

updated How to compile the OmniROM 14 for the the ASUS Zenfone 8 (see here or here for details)

21.01.2024

updated How to compile the OmniROM 14 for the the ASUS Zenfone 8 (see here)

19.01.2024

updated the section Some hints about the Android safe mode
updated the section How to install and configure the Android OS

18.01.2024

updated the section How to enable adb via WiFi
updated and published the section How to apply the Android Security patches to a local repository

17.01.2024

updated the section How to apply the Android Security patches to a local repository
added the section How to enable adb via WiFi

14.01.2024

added the section How to apply the Android Security patches to a local repository

13.01.2024

added the section How to compile the image for TWRP 3.7.0.12-1 w/ the bootctl binary

12.01.2024

updated the section How to switch the active slot
updated the section How to install and configure the Android OS
fixed typos in various sections

11.01.2024

updated the section How to add additional files to an TWRP image

10.01.2024

updated How to compile the OmniROM 14 for the the ASUS Zenfone 8 (see here)
added a link to a TWRP image for the ASUS Zenfone 8 running the LineageOS 20
updated the section How to use TWRP if LineageOS 20.x is installed

09.01.2024

updated How to compile the OmniROM 14 for the the ASUS Zenfone 8 (see here)

03.01.2024

added the section Links for MIcroG

02.01.2024

added the section How to enable and configure the Magisk DenyList via cli commands

26.12.2023

updated the section How to get or set the lock screen settings via CLI command (see here)

25.12.2023

updated the section How to get or set the lock screen settings via CLI command (see here)

22.12.2023

added the section How to get or set the lock screen settings via CLI command

18.12.2023

added infos about how to install "outdated" apks in Android 14 to the section How to install packages (apk files) for Android via script

26.11.2023

added the section How to compile the OmniROM 14 for the the ASUS Zenfone 8
added the TWRP images for the ASUS Zenfone 8 with Android 14 support (see below)

10.10.2023

Corrected all links to the XDA forum to use the new URL https://xdaforums.com
Corrected some not working links
added the section How to install the Android OS and Magisk via script and also enable access via adb

25.09.2023

added the section How to analyze the boot loader of an ASUS Zenfone 8

23.09.2023

updated the list of links for the ASUS Zenfone 8

11.09.2023

added the section How to configure the WiFI in Android via CLI command

26.08.2023/bs

added a link to an XDA Forum entry with a list of firmware image availabe on non-ASUS websites

17.08.2023 /bs

added the section ASUS Zenfone 8 FIrmware Download

11.07.2023 /bs

added more details about how to configure the SELinux context for the new file in the section How to change any file or directory using Magisk

08.07.2023 /bs

small error corrections in some of the howtos

30.06.2023 /bs

updated How to change the kernel boot parameter for the Android kernel

29.06.2023 /bs

published How to change the kernel boot parameter for the Android kernel

26.06.2023 /bs

published the section How to use TWRP if LineageOS 20.x is installed
updated the section Some hints for using TWRP (see here)

25.06.2023 /bs

added the section How to change the kernel boot parameter for the Android kernel
added the section How to use TWRP if LineageOS 20.x is installed

24.06.2023 /bs

added infos about using an overlayfs to make read-only filesystems writable to the section How to change any file or directory using Magisk (see here)

12.06.2023 /bs

finished and published the section How to temporary change a file on a read-only filesystem in the Android OS

11.06.2023/bs

updated the section How to install TWRP into the boot partition again after the installation of an OS update
added the section How to upgrade the OS with another Android "distribution"
added the section How to temporary change a file on a read-only filesystem in the Android OS

08.06.2023 /bs

updated the trouble shooting instruction in the section How to install a more recent version of the Playstore in OmniROM

05.06.2023 /bs

added a link to a Magisk module with another script to do the same to the section How to install TWRP into the boot partition again after the installation of an OS update
added the section How to install a more recent version of the Playstore in OmniROM

26.05.2023 /bs

added a small hint about the variable MAGISKTMP to the section Hints for adding files to the boot partition

24.05.2023 /bs

corrected some not working links
added the section Hints for adding files to the boot partition

19.05.2023 /bs

updated the section Some hints for using TWRP (see here)

16.05.2023 /bs

added the section How to install TWRP into the boot partition again after the installation of an OS update

12.05.2023 /bs

added the section How to re-install the Android OS from ASUS on the ASUS Zenfone 8
updated the section Some hints for using TWRP (see here)

07.05.2023 /bs

updated the section Some hints for using Magisk on Android phones - see also here.

06.05.2023 /bs

updated the section How to install Magisk into the boot partition using a script
added the version of the scripts in the list of available scripts
fixed some typos

30.04.2023 /bs

updated the section Some hints for using TWRP (see here)

29.04.2023/bs

corrected the URL because for the XDA post about How to install Magisk v26.0 or newer via script was moved by the forum admins

23.04.2023 /bs

updated the section Some hints for using TWRP (see here)
added the section How to recover data from a phone with non-working touch screen

18.04.2023 /bs

added the section How to install Magisk v26.0 or newer via script

13.04.2023 /bs

updated the section How to install Magisk into the boot partition using a script with hints about installing Magisk v26.x via script

15.03.2023 /bs

added the section Other Android related files

02.02.2023 /bs

added the section How to configure the WiFi in Android via script

27.01.2023 /bs

updated the section How to compile the OmniROM 13 for the ASUS Zenfone 8
updated and released the section How to install and configure the Android OS via script

18.01.2023 /bs

added the section How to install and configure the Android OS
added the section How to change any file or directory using Magisk

11.01.2023 /bs

updated the section How to enable access via adb on a new installed OS

01.01.2023 /bs

added the section Some hints about the Android safe mode

30.12.2022 /bs

updated the section Some hints for using Magisk on Android phones - see also here.
updated the section How to change files in the boot image using Magisk - see also here.

29.12.2022 /s

added the section How to enable access via adb on a new installed OS

27.12.2022 /bs

updated the section Some hints for using TWRP (see also here)
added the section How to fix the error Cannot load Android system. Your data may be corrupt.

19.12.2022 /bs

added the links to some distributions for the ASUS Zenfone 8

18.12.2022 /bs

added the section How to enable access via adb on a new installed OS
updated the section How to boot a phone from a TWRP recovery image via script
updated the section How to install Magisk into the boot partition using a script to match version 2.1.0.0 of the script
added the section How to install an OS image using the TWRP binary twrp

15.12.2022 /bs

updated the Magisk module addbin

07.12.2022 /bs

updated and published the section How to boot a phone from a TWRP recovery image via script

06.12.2022 /bs

added the section How to enable root access using Magisk in a script
updated the info about the touch screen bug the section TWRP related links
added the section Where does the Magisk App store the settings?

02.12.2022 /bs

updated the section How to boot a phone from a TWRP recovery image via script (the script was rewritten using functions now)
updated the section Some hints for using Magisk on Android phones - see also here.
added the section Some hints for using TWRP

27.11.2022/ bs

added the section How to compile TWRP for the ASUS Zenfone 8

26.11.2022 / bs

updated the section How to boot a phone from a TWRP recovery image via script

25.11.2022 /bs

updated the section Some hints for using Magisk on Android phones - see also here.

23.11.2022 /bs

updated the section How to change files in Android XML Binary format

21.11.2022 /bs

added the section How to change files in Android XML Binary format

14.11.2022 /bs

added the section How to compile the OmniROM 13 for the ASUS Zenfone 8

08.11.2022/bs

updated the section Some hints for using Magisk on Android phones - see also here.

07.11.2022/bs

added the section How to restore the boot partition to remove Magisk
updated the section Some hints for using Magisk on Android phones - see also here.
all links to the other posts in the posts are now links to the section in this page

06.11.2022 /bs

added the section How to boot a phone from a TWRP recovery image via script
updated the section How to switch the active slot
updated the section Some hints for using Magisk on Android phones - see also here.

04.11.2022/bs

added the section How to make files in /system writable
added the section HowTo Overview
updated the section Some hints for using Magisk on Android phones - see also here.

03.11.2022 /bs

updated the section How to compile the OmniROM for the ASUS Zenfone 8

02.11.2022 /bs

added the section How to process boot image files on a PC running the Linux OS

29.10.2022 /bs

added the section How to add additional files to an TWRP image
corrected invalid URLs in the section with the URLs

28.10.2022 /bs
added the section How to install TWRP via script
added the section TWRP related links
updated the section Some hints for using Magisk on Android phones - see also here.
updated the section How to install Magisk into the boot partition using a script to match version 2.0.0 of the script

04.10.2022 /bs

added the section How to trigger an action when a property is changed

02.10.2022 /bs

added the section How to run a script at shutdown

30.09.2022 /bs

added infos about how to change files in the root filesystem using Magisk to the section How to change files in the boot image using Magisk
updated the section Some hints for using Magisk on Android phones - see also here.

29.09.2022 /bs

updated the section How to compile the OmniROM for the ASUS Zenfone 8
added the section How to switch the active slot

20.09.2022 /bs

added the section How to change files in the boot image using Magisk
added the section How to disable or change the swap device in the Android 12 from ASUS for the Zenfone 8
added the section How to create a swap device in the OmniROM 12 using Magisk
added the section Misc Links

07.08.2022 /bs

updated the section Some hints for using Magisk on Android phones - see also here.
updated the section with links

05.08.2022/bs

added the section with link to various tools

04.08.2022 /bs

added test instructions to Instructions to use the ASUS Hardware Testtool

23.07.2022 /bs

added the section Some hints for using Magisk on Android phones

19.07.2022 /bs

updated the section with links

16.07.2022 /bs

added the section How to change the home directory for the user root on an Android phone
fixed some non-working links

05.07.2022 /bs

added the section How to install packages (apk files) for Android via script
added the section How to replace the Fake Store from OmniROM with MicroG with a patched Playstore

04.07.2022 /bs

updated the script to install Magisk
added the section with Links

28.06.2022 /bs

added the section How to backup the data from the phone using rsync and ssh (including some hints for using sshd on an Android phone)

26.06.2022 /bs

minor fixes in the Instructions to use the ASUS Hardware Testtool

25.06.2022 /bs

initial release

HowTos

Published Howtos Overview

List of URLs for all also elsewhere published HowTos sorted by type of the HowTo

Android related articles

How to install packages (apk files) for Android via script

https://xdaforums.com/t/how-to-install-packages-apk-files-for-android-via-script.4464987/

How to trigger an action when a property is changed

https://xdaforums.com/t/how-to-trigger-an-action-when-a-property-is-changed.4501489/

How to backup the data from the phone using rsync and ssh (including some hints for using sshd on an Android phone)

https://xdaforums.com/t/how-to-backup-the-data-from-the-phone-using-rsync-and-ssh-including-some-hints-for-using-sshd-on-an-android-phone.4462327/

How to change the home directory for the user root on an Android phone

https://xdaforums.com/t/how-to-change-the-home-directory-for-the-user-root-on-an-android-phone.4469305/

How to process Android boot image files on a PC running the Linux OS

https://xdaforums.com/t/how-to-process-android-boot-image-files-on-a-pc-running-the-linux-os.4513845/

How to change files in Android XML Binary format

https://xdaforums.com/t/how-to-change-files-in-android-xml-binary-format.4521513/

How to fix the error "Cannot load Android system. Your data may be corrupt".

https://xdaforums.com/t/how-to-fix-the-error-cannot-load-android-system-your-data-may-be-corrupt.4534237/

How to enable access via adb on a new installed OS

https://xdaforums.com/t/how-to-enable-access-via-adb-on-a-new-installed-os.4535165/

Some hints about the Android safe mode

https://xdaforums.com/t/some-hints-about-the-android-safe-mode.4536269/

How to install and configure the Android OS via script

https://xdaforums.com/t/how-to-install-and-configure-the-android-os-via-script.4546375/

How to install the Android OS and Magisk via script and also enable access via adb

https://xdaforums.com/t/how-to-enable-access-via-adb-on-a-new-installed-os.4535165/#post-89088101

How to configure the WiFI in Android via Script

https://xdaforums.com/t/how-to-configure-the-wifi-in-android-via-script.4548489/

How to configure the WiFI in Android via cli command

https://xdaforums.com/t/how-to-configure-the-wifi-in-android-via-cli-command.4626435/

How to upgrade the OS with another Android "distribution"

https://xdaforums.com/t/how-to-upgrade-the-os-with-another-android-distribution.4595049/

How to temporary change a file on a read-only filesystem in the Android OS

https://xdaforums.com/t/how-to-temporary-change-a-file-on-a-read-only-filesystem-in-the-android-os.4595429/

How to change the kernel boot parameter for the Android kernel

https://xdaforums.com/t/how-to-change-the-kernel-boot-parameter-for-the-android-kernel.4600821/

How to add files to the ramdisk used for the recovery boot

https://xdaforums.com/t/how-to-add-files-to-the-ramdisk-used-for-the-recovery-boot.4678287/

How to get or set the lock screen settings via CLI command

https://xdaforums.com/t/how-to-get-or-set-the-lock-screen-settings-via-cli-command.4647038/

How to enable adb via WiFi

https://xdaforums.com/t/how-to-enable-adb-via-wifi.4651610/

How to get the application name and version from an apk file

https://xdaforums.com/t/how-to-get-the-application-name-and-version-from-an-apk-file.4654732/

How to compile a C program for Android using the Android NDK CLI tools

https://xdaforums.com/t/how-to-compile-a-c-program-for-android-using-the-android-ndk-cli-tools.4656126/

Compiling C programs in Android

https://xdaforums.com/t/how-to-compile-c-programs-in-android.4683687/

Compiling C programs in Android using gcc

https://xdaforums.com/t/guide-compiling-c-programs-in-android-using-gcc.4695529/

How to compile Perl for Android

https://xdaforums.com/t/how-to-compile-c-programs-in-android.4683687/#post-89651799

How to get the security patch level in an ROM image zip file

https://xdaforums.com/t/how-to-get-the-security-patch-level-used-in-a-rom-image-zip-file.4666081/

How to add missing SELinux policies dynamically using Magisk

https://xdaforums.com/t/how-to-add-missing-selinux-policies-dynamically-using-magisk.4671716/

How to re-sign an apk file

https://xdaforums.com/t/how-to-re-sign-an-apk-file.4671724/

How to "simulate" prebuild apps using Magisk

https://xdaforums.com/t/how-to-simulate-prebuild-apps-using-magisk.4671914/

How to reset the USB port used to connect an Android phone

https://xdaforums.com/t/how-to-reset-the-usb-port-used-to-connect-an-android-phone.4672845/

Some hints about adb and fastboot usage in Linux

https://xdaforums.com/t/some-hints-about-adb-and-fastboot-usage-in-linux.4673460/

How to automatically start adbd as root user after starting the phone

https://xdaforums.com/t/how-to-automatically-start-adbd-as-root-user-after-starting-the-phone.4682701/

How to list bind mounts

https://xdaforums.com/t/how-to-list-bind-mounts.4684276/

How to create logical devices using the device mapper from Android

https://xdaforums.com/t/guide-how-to-create-logical-devices-using-the-device-mapper-from-android.4690991/

How to create a new dynamic partition in the partition super

https://xdaforums.com/t/guide-how-to-create-a-new-dynamic-partition-in-the-partition-super.4691372/

How to create logical devices for disk image files

https://xdaforums.com/t/guide-how-to-create-logical-devices-for-disk-image-files.4691642/

Magisk related articles

Some hints for using Magisk on Android phones

https://xdaforums.com/t/some-hints-for-using-magisk-on-android-phones.4471857/

Where does the Magisk App store the settings?

https://xdaforums.com/t/where-does-the-magisk-app-store-the-settings.4527129/

How to install Magisk into the boot partition using a script

https://xdaforums.com/t/how-to-install-magisk-into-the-boot-partition-using-a-script.4456621/

How to change files in the directory /system with Magisk

https://xdaforums.com/t/how-to-change-files-in-the-directory-system-with-magisk.4454489/

How to change any file or directory using Magisk

https://xdaforums.com/t/how-to-change-any-file-or-directory-using-magisk.4543103/

How to run a script at every boot using Magisk

https://xdaforums.com/t/how-to-run-at-script-a-every-boot-using-magisk.4454493/

How to change files in the boot image using Magisk

https://xdaforums.com/t/how-to-change-files-in-the-boot-image-using-magisk.4495645/

How to changes files in the boot image using Magisk if there is no ramdisk in the boot partition

https://xdaforums.com/t/how-to-change-files-in-the-boot-image-using-magisk-if-there-is-no-ramdisk-in-the-boot-partition.4678017/

Hints for adding files to the boot partition

https://xdaforums.com/t/how-to-change-files-in-the-boot-image-using-magisk.4495645/#post-88571069

How to run a script at shutdown

https://xdaforums.com/t/how-to-run-a-script-at-shutdown.4500719/

How to restore the boot partition to remove Magisk

https://xdaforums.com/t/how-to-restore-the-boot-partition-to-remove-magisk.4515759/

How to enable root access using Magisk in a script

https://xdaforums.com/t/how-to-enable-root-access-using-magisk-in-a-script.4527035/

How to install Magisk v26.0 or newer via script

https://xdaforums.com/t/how-to-install-magisk-into-the-boot-partition-using-a-script.4456621/post-88429939

How to enable and configure the Magisk DenyList via cli commands

https://xdaforums.com/t/how-to-enable-and-configure-the-magisk-denylist-via-cli-commands.4648761/

Some hints for writing init scripts for Magisk

https://xdaforums.com/t/guide-some-hints-for-writing-init-scripts-for-magisk.4689292/

How to install Magisk Modules via script

https://xdaforums.com/t/guide-how-to-install-magisk-modules-via-a-script.4694326/

Magisk Module with OpenSSH for Android

https://xdaforums.com/t/module-openssh-for-android.4696499/

Magisk Module with network tools for CLI sessions

https://xdaforums.com/t/module-network-tools-for-cli-sessions.4696517/

Magisk Module with Python

https://xdaforums.com/t/module-python-3-14-for-android.4696862/

TWRP related articles

Some hints for using TWRP

https://xdaforums.com/t/some-hints-using-twrp.4525833/

How to install TWRP via script

https://xdaforums.com/t/how-to-install-twrp-via-script.4511355/

How to add additional files to an TWRP image

https://xdaforums.com/t/how-to-add-additional-files-to-an-twrp-image.4511827/

How to compile TWRP for the ASUS Zenfone 8

https://xdaforums.com/t/how-to-compile-twrp-for-the-asus-zenfone-8.4523817/

How to compile the image for TWRP 3.7.0.12-1 w/ the bootctl binary

https://xdaforums.com/t/how-to-compile-the-image-for-twrp-3-7-0-12-1-w-the-bootctl-binary.4650684/

How to boot a phone from a TWRP recovery image via script

https://xdaforums.com/t/how-to-boot-a-phone-from-a-twrp-recovery-image-via-script.4527547/

How to install an OS image using the TWRP binary twrp

https://xdaforums.com/t/how-to-install-an-os-image-using-the-twrp-binary-twrp.4531345/

How to install an OS image via sideload using the TWRP binary twrp

https://xdaforums.com/t/how-to-install-an-os-image-via-sideload-using-the-twrp-binary-twrp.4669384/

How to recover data from a phone with non-working touch screen

https://xdaforums.com/t/how-to-recover-data-from-a-phone-with-a-non-working-touch-screen.4578795/

How to install TWRP into the boot partition again after the installation of an OS update

https://xdaforums.com/t/installing-twrp-into-the-boot-partition-again-after-the-installation-of-an-os-update.4586725/

OrangeFox related articles

How to compile the OrangeFox recovery for the ASUS Zenfone 8

https://xdaforums.com/t/how-to-compile-the-orangefox-recovery-for-the-asus-zenfone-8.4692137/

OmniROM related articles

How to compile the OmniROM for the ASUS Zenfone 8

https://xdaforums.com/t/how-to-compile-the-omnirom-for-the-asus-zenfone-8.4461525/

How to compile the OmniROM 13 for the ASUS Zenfone 8

https://xdaforums.com/t/how-to-compile-the-omnirom-13-for-the-android-zenfone-8.4518733/

How to compile the OmniROM 14 for the ASUS Zenfone 8

https://xdaforums.com/t/how-to-compile-omnirom-14-for-the-asus-zenfone-8.4643071

Infos for building an Android 14 based CustomROM image in Q1 2024 or later

https://xdaforums.com/t/infos-for-building-an-android-14-based-customrom-image-in-q1-2024-or-later.4664652/

How to replace the Fake Store from OmniROM with MicroG with a patched Playstore

https://xdaforums.com/t/magisk-module-to-replace-the-fakestore-in-the-omnirom-with-a-patched-playstore-from-nanodroid.4456421/

How to install a more recent version of the Playstore in OmniROM

https://xdaforums.com/t/how-to-install-a-more-recent-version-of-the-playstore-in-the-omnirom.4593203/

How to create a swap device in the OmniROM 12 using Magisk

https://xdaforums.com/t/how-to-create-or-change-a-swap-device-in-the-omnirom-using-magisk.4495415/

How to apply the Android Security patches to a local repository

https://xdaforums.com/t/how-to-apply-the-android-security-patches-to-a-local-repository.4651526/

LineageOS related articles

How to use TWRP if LineageOS 20.x is installed

https://xdaforums.com/t/how-to-use-twrp-if-lineageos-20-x-is-installed.4599721/

How to create a TWRP image for LineageOS based ROMs using a script

https://xdaforums.com/t/how-to-use-twrp-if-lineageos-20-x-is-installed.4599721/#post-89582817

How to access the system partition while the phone is booted into the LineageOS recovery

https://xdaforums.com/t/guide-how-to-access-the-system-partition-while-the-phone-is-booted-into-the-lineageos-recovery.4691374/

General ROM Development related articles

Some hints for repo sync troubleshooting

https://xdaforums.com/t/some-hints-for-trouble-shooting-repo-sync-issues.4655310/

Some hints for fixing errors building an Custom ROM Android image

https://xdaforums.com/t/some-hints-for-fixing-errors-building-an-custom-rom-android-image.4655328/

How to add additional prebuild apks to a custom ROM

https://xdaforums.com/t/how-to-add-additional-prebuild-apks-to-a-custom-rom.4671911/

How to apply an Android patchlevel to the repositories for the StatixOS

https://xdaforums.com/t/how-to-apply-an-android-security-patchlevel-to-a-local-repository-for-the-statixos.4686280

ASUS Zenfone 8 related articles

How to use the Hardware Test App from ASUS on a Zenfone running a CustomROM

https://xdaforums.com/t/how-to-use-the-hardware-test-app-from-asus-on-a-zenfone-running-a-customrom.4461533/

How to disable or change the swap device in the Android 12 from ASUS for the Zenfone 8

https://xdaforums.com/t/how-to-disable-or-change-the-swap-device-in-the-android-12-from-asus-for-the-zenfone-8.4495085/

How to re-install the Android OS from ASUS on the ASUS Zenfone 8

https://xdaforums.com/t/asus-zenfone-8-how-to-rollback-from-lineageos-to-original-rom.4583833/#post-88525935

Download firmware for the ASUS Zenfone 8

https://xdaforums.com/t/zenfone-8-need-android-12-firmware-to-downgrade-from-android-13.4617265/#post-88887299

How to analyze the boot loader of an ASUS Zenfone 8

https://xdaforums.com/t/how-to-analyze-the-boot-loader-of-an-asus-zenfone-8.4631111/

How to add the ASUS Hardware TestTool as prebuild app for a Custom ROM

https://xdaforums.com/t/how-to-add-the-asus-hardware-testtool-as-prebuild-app-for-a-custom-rom.4671918/

How to add the ASUS Hardware Testtool as prebuild app to the StatiXOS

https://xdaforums.com/t/how-to-add-the-asus-hardware-testtool-as-prebuild-app-for-a-custom-rom.4671918/#post-89528935

General articles

How to switch the active slot

https://xdaforums.com/t/how-to-manually-switch-the-active-slot.4499789/

How to print the security patch level used in repositories or ROM image zip files

https://xdaforums.com/t/how-to-print-the-security-patch-level-used-in-repositories-or-rom-image-zip-files.4666170/

How to fix a hanging restore in Titanium Backup

https://xdaforums.com/t/how-to-fix-a-hanging-restore-in-titanium-backup.4666620/

Some hints about EDL mode

https://xdaforums.com/t/some-hints-about-edl-mode.4672703/

Not yet published HowTos

How to compile Magisk

How to compile a C program for Android (not yet published, but you can find a reference to this entry here: https://xdaforums.com/t/how-to-compile-a-c-program-for-android-using-the-android-ndk-cli-tools.4656126/)

How to compile a C program for Android using gcc (not yet published, but you can find a reference to this entry here: https://xdaforums.com/t/how-to-compile-a-c-program-for-android-using-the-android-ndk-cli-tools.4656126/)

Examples for compiling C programs for Android using gcc or clang (not yet published, but you can find a reference to this entry here: https://xdaforums.com/t/how-to-compile-a-c-program-for-android-using-the-android-ndk-cli-tools.4656126/)

How to install and configure other ROMs via script (not yet published, but you can find a reference to this entry here: https://xdaforums.com/t/how-to-install-and-configure-the-android-os-via-script.4546375/#post-89489846)

Observations while checking how to apply the Android Securiiy patches to a local repository

How to sign a zip file with a Custom ROM image

How to boot the phone into the bootloader using the physical buttons

How to install "fastboot images"

Some hints for SELinux usage in Android (not yet published, but you can find a reference to this entry here: https://xdaforums.com/t/some-hints-for-selinux-usage-in-android.4683677/)

How to start or stop an app via CLI command

How to use a log host within the Android operating system

Some hints for creating Magisk Modules

---

Most of the HowTos are listed below in the chronological order in which I created them.

How to compile the OmniROM for the ASUS Zenfone 8

URL: https://xdaforums.com/t/how-to-compile-the-omnirom-for-the-asus-zenfone-8.4461525/

How to compile the OmniROM for the ASUS Zenfone 8

Update 25.11.2023 /bs

see How to compile the OmniROM 14 for the the ASUS Zenfone 8 for how to compile the OmniROM 14 for the ASUS Zenfone 8 and some more details regarding the build tools and the build process.

Update 14.11.2022 /bs

This HowTo is about how to compile the OmniROM 12 for the ASUS Zenfone 8 -- see How to compile the OmniROM 13 for the ASUS Zenfone 8 for how to compile the OmniROM 13 for the ASUS Zenfone 8.

Correction

There was a spelling error in the name of the command to list all available targets : the command name is breakfast

Compiling the CustomROM OmniROM from within Linux is a very simple and straight forward process -- the developer for the OmniROM did a really amazing good job setting up the build environment for the OmniROM.

But some information are missing in the instructions (at least I did not found them ...)

Therefor I document here what I did to compile the OmniROM for the ASUS Zenfone 8 on my Linux notebook:

First install the neccessary tool called "repo" to maintain the repositories for the OmniROM ; see here https://gerrit.googlesource.com/git-repo/

Next create a new directory on your PC; switch to that directory, and initialize the necessarry repositories like described in the section "Getting Started" here: https://github.com/omnirom/android :

Note [29.07.2022]:

The repositories for the branch android-12.0 are somehow outdated with the security patch level from February 2022 . To get the code with the security patch level from June 2022 use the repositories for the branch android-12.1.



        repo init -u https://github.com/omnirom/android.git -b
        android-12.1

Then to sync up:

repo sync

Be aware that the repositories need about 140 GB disk space so this will take a while.

Please note that I did not install any additional tool to compile the OmniROM - everything neccessary was installed by synching the repositories.

When the download is finished you must add the repositories for the vendor and device tree for your phone to the manifests in the local repositories. This should be done in the file .repo/local_manifests/local_manifest.xml . For the ASUS Zenfone 8 the file should look like this:

[xtrnaw7@t15g /data/develop/android/OmniROM]$ cat
        .repo/local_manifests/local_manifest.xml

        <?xml version="1.0" encoding="UTF-8"?>

        

        <manifest>

        <remote name="DonkeyCoyote"

        fetch="https://github.com/DonkeyCoyote" />

        

        <project path="device/asus/zenfone8"
        name="android_device_asus_zenfone8" remote="omnirom"
        revision="android-12.0" />

        <project path="vendor/asus" name="proprietary_vendor_asus"
        remote="DonkeyCoyote" revision="android-12.1" />

        </manifest>

After updating the file local_manifest.xml do another

repo sync

to sync the added repositories. This time the download will be faster because only the two additional repos will be downloaded.

Now you can start the build with these commands:

. build/envsetup.sh 

        

        brunch omni_zenfone8-user

in the directory with the downloaded repositories (use the command "breakfast" instead of "brunch" to see other possible targets)

Note that the first time the compile process will take 4 hours or more. And keep in mind that the compile process will need additional 140 GB for the temporary output files.

If everything is fine there will be a ZIP file with your own CustomROM image in the directory ./out/target/product/zenfone8/ when the compile script is done, e.g:

[xtrnaw7@t15g /data/develop/android/OmniROM]$ ls -ltr
out/target/product/zenfone8/omni-12-20220621-zenfone8-HOMEMADE.zip

         -rw-rw-r--. 25 xtrnaw7 xtrnaw7 1253703465 Jun 25 10:03
        out/target/product/zenfone8/omni-12-20220621-zenfone8-HOMEMADE.zip
        

        [xtrnaw7@t15g /data/develop/android/OmniROM]$

That image can now be installed on the Zenfone 8 using the LineageOS Recovery or TWRP.

Update 29.09.2022 /bs

There seems to be a bug or feature in the scripts to compile OmniROM:

Old versions of the ZIP file with the image in the output directory will always be overwritten or replaced with the current image, e.g.:

[ OmniRomDev - xtrnaw7@t15g /data/develop/android/test ] $ ls -l /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220*zip -rw-rw-r--. 20 xtrnaw7 xtrnaw7 1256445293 Sep 27 18:12 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220728-zenfone8-MICROG.zip -rw-rw-r--. 20 xtrnaw7 xtrnaw7 1256445293 Sep 27 18:12 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220729-zenfone8-MICROG.zip -rw-rw-r--. 20 xtrnaw7 xtrnaw7 1256445293 Sep 27 18:12 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220804-zenfone8-MICROG.zip -rw-rw-r--. 20 xtrnaw7 xtrnaw7 1256445293 Sep 27 18:12 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220809-zenfone8-HOMEMADE.zip -rw-rw-r--. 20 xtrnaw7 xtrnaw7 1256445293 Sep 27 18:12 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220810-zenfone8-MICROG.zip -rw-rw-r--. 20 xtrnaw7 xtrnaw7 1256445293 Sep 27 18:12 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220820-zenfone8-MICROG.zip -rw-rw-r--. 20 xtrnaw7 xtrnaw7 1256445293 Sep 27 18:12 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220828-zenfone8-MICROG.zip -rw-rw-r--. 20 xtrnaw7 xtrnaw7 1256445293 Sep 27 18:12 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220904-zenfone8-MICROG.zip -rw-rw-r--. 20 xtrnaw7 xtrnaw7 1256445293 Sep 27 18:12 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220911-zenfone8-MICROG.zip -rw-rw-r--. 20 xtrnaw7 xtrnaw7 1256445293 Sep 27 18:12 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220918-zenfone8-MICROG.zip -rw-rw-r--. 20 xtrnaw7 xtrnaw7 1256445293 Sep 27 18:12 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220927-zenfone8-MICROG.zip [ OmniRomDev - xtrnaw7@t15g /data/develop/android/test ] $ [ OmniRomDev - xtrnaw7@t15g /data/develop/android/test ] $ cksum /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220*zip 592627598 1256445293 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220728-zenfone8-MICROG.zip 592627598 1256445293 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220729-zenfone8-MICROG.zip 592627598 1256445293 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220804-zenfone8-MICROG.zip 592627598 1256445293 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220809-zenfone8-HOMEMADE.zip 592627598 1256445293 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220810-zenfone8-MICROG.zip 592627598 1256445293 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220820-zenfone8-MICROG.zip 592627598 1256445293 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220828-zenfone8-MICROG.zip 592627598 1256445293 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220904-zenfone8-MICROG.zip 592627598 1256445293 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220911-zenfone8-MICROG.zip 592627598 1256445293 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220918-zenfone8-MICROG.zip 592627598 1256445293 /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20220927-zenfone8-MICROG.zip [ OmniRomDev - xtrnaw7@t15g /data/develop/android/test ]

Update 08.02.2024

see Some hints for repo sync troubleshooting for how to fix common errors executing repo sync.

see Some hints for fixing errros building an Custom ROM Android image for how to fix some common errors while creating the image

How to create an OmniROM image with MicroG

How to create an OmniROM image with MicroG

To create an OmniROM image with MicroG these additional steps are neccesarry:

Add the repo with the prebuild MicroG apks, android_prebuilts_prebuiltapks, to your local_manifests.xml file, e.g:

[xtrnaw7@t15g /data/develop/android/OmniROM]$ cat
        .repo/local_manifests/local_manifest.xml

        <?xml version="1.0" encoding="UTF-8"?>

        <manifest>

        <remote name="DonkeyCoyote"

        fetch="https://github.com/DonkeyCoyote" />

        <project path="device/asus/zenfone8"
        name="android_device_asus_zenfone8" remote="omnirom"
        revision="android-12.0" />

        <project path="vendor/asus" name="proprietary_vendor_asus"
        remote="DonkeyCoyote" revision="android-12.1" />

        <project path="vendor/microg"
          name="android_prebuilts_prebuiltapks" remote="omnirom"
          revision="android-12.0" />

        </manifest>

        [xtrnaw7@t15g /data/develop/android/OmniROM]$

Do another repo sync to download the repository for MicroG:



        repo sync

and create the CustomROM using these instructions:



         export ROM_BUILDTYPE=MICROG 

        

        brunch omni_zenfone8-user

This creates an OmniROM Image with MicroG already installed, e.g:

[xtrnaw7@t15g /data/develop/android/OmniROM]$ ls -ltr
        out/target/product/zenfone8/omni-12-20220621-zenfone8-MICROG.zip
        

        -rw-rw-r--. 25 xtrnaw7 xtrnaw7 1253703465 Jun 25 10:03
        out/target/product/zenfone8/omni-12-20220621-zenfone8-MICROG.zip
        

        [xtrnaw7@t15g /data/develop/android/OmniROM]$

How to use the Hardware Test App from ASUS on a Zenfone running a CustomROM

URL: https://xdaforums.com/t/how-to-use-the-hardware-test-app-from-asus-on-a-zenfone-running-a-customrom.4461533/

How to use the Hardware Test App from ASUS on a Zenfone running a CustomROM

The Hardware Test App from ASUS for the ASUS Zenfone 8 can also be used on a ASUS Zenfone 8 running a CustomROM.

But to get the App working on a CustomROM some efforts are neccessary.

The ASUS Hardware Test App needs the permission to modify system settings. To get this permission the app must be signed with the platform certificate used for the running Android OS.
And because these certificates are not available for the public for all public available CustomROMs (for some very good reasons ...) you must compile your own CustomROM for using the ASUS Hardware Test App on a CustomROM.

So -- for those who still want to continue: Here are the steps neccessary to run the Hardware Test App from ASUS for the Zenfone 8 on a phone running a CustomROM :

First copy the apk with the ASUS Hardware Test App from a Zenfone 8 running the original Android 12 from ASUS to your PC.

This is the file

/system/apps/SMMI_TEST/SMMI_TEST.apk

on the phone running the ASUS Android OS.

Now it's neccessary to sign the App with the platform key from your CustomROM:

In the OmniROM (and I assume that's also true for other AOSP based CustomROMs) the neccessary files, platform.pk8 and platform.x509.pem, are in the directory

./build/make/target/product/security

in your build tree for the CustomROM, e.g.

[xtrnaw7@t15g /data/develop/android/OmniROM]$ ls -l
          ./build/make/target/product/security/platform.*

        -rw-------. 1 xtrnaw7 xtrnaw7 1219 Jun 25 09:39
        ./build/make/target/product/security/platform.pk8

        -rw-rw-r--. 1 xtrnaw7 xtrnaw7 1460 Jun 25 09:39
        ./build/make/target/product/security/platform.x509.pem

        [xtrnaw7@t15g /data/develop/android/OmniROM]$

Now re-sign the apk file using these commands:



        # remove the current certificate files from the apk (probably
        not neccessary -- but shouldn't harm)

        #

        zip -d SMMI_TEST.apk META-INF/CERT.SF META-INF/CERT.RSA

        

        # do a zip align for the apk and write the output to
        SMMI_TEST1.apk

        #

        # (zipalign is part of the OTA tools)

        #

        /data/develop/android/otatools/bin/zipalign 4 SMMI_TEST.apk
        SMMI_TEST1.apk

        

        

        # test the result (there should be NO output from this command)

        #

        /data/develop/android/otatools/bin/zipalign -c 4 SMMI_TEST1.apk

        

        

        # and now sign the apk with the platform key from the
        development tree for your self compiled CustomROM

        # (apksigner.jar is part of the Sdk from Google for Android)

        #

        java -jar ./Android/Sdk/build-tools/33.0.0/lib/apksigner.jar
        sign --key platform.pk8 --cert platform.x509.pem SMMI_TEST1.apk

That's it . The re-signed apk file SMMI_TEST1.apk should work on your CustomROM

To test the signed apk do:

Copy the signed apk to the phone running your self compiled CustomROM and issue

ls -l /sdcard/Download/SMMI_TEST1.apk

cat /sdcard/Download/SMMI_TEST1.apk | pm install -S 25733396

If you get an error like this

ASUS_I006D:/ # ls -l /sdcard/Download/SMMI_TEST1.apk -rw-rw---- 1 u0_a111 media_rw 25733396 2022-08-04 19:02 /sdcard/Download/SMMI_TEST1.apk ASUS_I006D:/ # ASUS_I006D:/ # cat /sdcard/Download/SMMI_TEST1.apk | pm install -S 25733396 Failure [INSTALL_FAILED_SHARED_USER_INCOMPATIBLE: Reconciliation failed...: Reconcile failed: Package com.asus.atd.smmitest has no signatures that match those in shared user android.uid.system; ignoring!] 1|ASUS_I006D:/ #

something went wrong signing the apk. Just do it again.

To install the apk do:

If not already done:

Compile your own CustomROM (using the certificates used to sign the ASUS Hardware Test App!) and install it on the ASUS Zenfone 8. (see How to compile the OmniROM for the ASUS Zenfone 8 for how to compile the OmniROM)

Install Magisk on your phone running the self compiled CustomROM

Simulate a Magisk Module using these commands:

adb shell su - -c mkdir -p
        /data/adb/modules/SMMI_TEST/system/app/SMMI_TEST

        

        adb push SMMI_TEST1.apk /sdcard/Download/

        

        adb shell su - -c cp /sdcard/Download/SMMI_TEST1.apk
        /data/adb/modules/SMMI_TEST/system/app/SMMI_TEST/SMMI_TEST1.apk

        

        adb shell su - -c chmod o+r
        /data/adb/modules/SMMI_TEST/system/app/SMMI_TEST/SMMI_TEST1.apk

After the reboot the ASUS Hardware Test App should be visible in the directory /system/app, e.g.



        

        ASUS_I006D:/ # find /system/app/SMMI_TEST/

        /system/app/SMMI_TEST/

        /system/app/SMMI_TEST/SMMI_TEST1.apk

        ASUS_I006D:/ #

        

        # open a shell on the ASUS Zenfone 8 (either local or via "adb
        shell"), become root user 

        

        su -

and execute



        # switch SELinux to permissive mode (I currently do not know the
        neccessary SELinux policies to avoid this step ...)

        #

        setenforce 0

        

        # and start the ASUS Hardware Test App from within the shell

        #

        am start -n
        com.asus.atd.smmitest/com.asus.atd.smmitest.main.MAIN

Do not forget to enable SELinux enforcing mode after the tests are done (or reboot the phone ...)

# switch SELinux to permissive mode (I currenlty do not know
        the neccessary SELinux policies to avoid this step ...) 

        # 

        setenforce

Sample Script to start the ASUS Hardware Test App

# simple script to start the ASUS Hardware App

        

        CUR_SELINUX_STATUS="$( getenforce )"

        CUR_USERID=$( id -u -n )

        RESTORE_SELINUX_STATE="false"

        

        if [ "${CUR_SELINUX_STATUS}"x != "Permissive"x ] ; then

        

        if [ "${CUR_USERID}"x != "root"x ] ; then

          echo "Either start this script as user \"root\" or set
        the SELinux mode to \"permissive\" using the command
        \"setenforce 0\" before starting this script"

          exit 1

        fi

        echo "Setting the SELinux mode to \"Permissive\" ..."

        setenforce 0

        RESTORE_SELINUX_STATE="true"

        else

          echo "The SELinux mode is already \"Permissive}\" "

        fi

        

        am start -n
        com.asus.atd.smmitest/com.asus.atd.smmitest.main.MAIN
        2>&1 | tee /sdcard/Download/test.out

        

        if [ ${RESTORE_SELINUX_STATE} = true ] ; then

          echo "Press return when done with the tests .."

          read USER_INPUT

          echo "Setting the SElinux mode back to \"Enforcing\"
        ...."

          setenforce 1

        else

          echo "Do not forget to set the SELinxu mode back to
        \"Enforcing\" using the command \"setenforce 1\" "

        fi

Trouble Shooting

If the boot process of the phone hangs after adding the pseudo Magisk Module for the ASUS Hardware Test App do:

# connect to the phone via adb (that should work!)

        #

        adb shell

        

        # become root user

        #

        su -

        

        # delete the file with the AUS Hardware Test App

        #

        rm
        /data/adb/modules/SMMI_TEST/system/app/SMMI_TEST/SMMI_TEST1.apk

        

        # and reboot the phone

        #

        reboot

Most probably something went wrong signing the app or you did not do the zip align (there should be some messages visible in logcat to find the reason for the boot loop; check the output of logcat before rebooting the phone)

A useful test to find reason for the boot loop is :

# try to install the apk as root user

        #

        su -

        

        # and then

        #

        setenforce 0

        pm install /sdcard/Download/SMMI_TEST.apk

If booting the phone works but the App does not start make sure that the SELinux status is permissive.

Update 14.05.2024

The necessary SELinux permissions for the test app are:

-(allow system_app_30_0 proc_stat_30_0 (file (ioctl read getattr lock map open watch watch_reads)))

To add these SELinux permissions the Magisk binary magiskpolicy can be used like this (as user root):

magiskpolicy --live "allow system_app proc_stat file ioctl" magiskpolicy --live "allow system_app proc_stat file read" magiskpolicy --live "allow system_app proc_stat file getattr" magiskpolicy --live "allow system_app proc_stat file lock" magiskpolicy --live "allow system_app proc_stat file map" magiskpolicy --live "allow system_app proc_stat file open" magiskpolicy --live "allow system_app proc_stat file watch" magiskpolicy --live "allow system_app proc_stat file watch_reads"

SELinux permissions configured using magiskpolicy are only temporary and will not survive a reboot. To permanent configure these SELinux policies create an init script for Magisk in the directory /data/adb/service d with the magiskpolicy commands.

It's also possible to install the ASUS Calculator on the phone via a Magisk Module so that the Test app can be executed by entering ".12345+=" in the calculator. Note that the calculator app must be installed as privileged app on the phone in the directory /system/priv-app//AsusCalculator. And the apk file with the calculator must also be signed with the certificate used for the ROM.

To retrieve the apk file for the ASUS Calculator issue :

adb pull /system/priv-app/AsusCalculator/AsusCalculator.apk
while the original Android from ASUS is running on the phone.

see also How to add missing SELinux policies dynamically using Magisk

see How to add additional prebuild apks to a custom ROM and How to add the ASUS Hardware Testtool as prebuild app for a Custom ROM for how to add the test app as prebuild app to a custom ROM

How to run a script at every boot using Magisk

URL: https://xdaforums.com/t/how-to-run-at-script-a-every-boot-using-magisk.4454493/

How to run at script a every boot using Magisk

Note:

I tested the instructions below with Magisk versions 24.3, 25.x, 26.x and 27.x . They may or may not work with other versions of Magisk

Another very useful feature from Magisk that can also be used without creating a Magisk Module is the ability to start scripts after booting the phone:

To execute scripts after rebooting the phone just copy them to one of the directories

/data/adb/post-fs-data.d
/data/adb/service.d

on the phone, make the script executable (

chmod +x
        scriptname

) and reboot the phone.

(See also https://github.com/topjohnwu/Magisk/blob/master/docs/guides.md#boot-scripts)

Because the mapping will be done while booting the phone it will also survive updating the OS on the phone to a new version as long as the user data is not deleted and Magisk is installed.

To remove the script just delete the script and reboot the phone.

In case the phone does not boot anymore after adding the script just boot the phone from a recovery with adb support, for example from the TWRP:

# boot the phone into the bootloader and then do 

        

        sudo fastboot boot
        /data/backup/ASUS_ZENFONE8/twrp/twrp-3.6.1_12-1-I006D.img

Then connect via "adb shell" to the phone and delete the script .

I used this method to correct the SELinux context for the NFC device in the first version of AospExtended 9.0 for the ASUS Zenfone 8 (the device tree is recreated from scratch each time the phone boots so the change must be done after every reboot):

cat /data/adb/service.d/correct_dev_pn553.sh

        #!/system/bin/sh

        

        NFC_DEVICE="/dev/pn553"

        

        echo ""

        echo "The SELinux context for \"${NFC_DEVICE}\" is now:"

        ls -lZ ${NFC_DEVICE}

        echo ""

        echo "Correcting the SELinux context for the NFC device
        \"${NFC_DEVICE}\" ..."

        chcon -v u:object_r:nfc_device:s0 /dev/pn553

        echo ""

        echo "The SELinux context for \"${NFC_DEVICE}\" is now:"

        ls -lZ ${NFC_DEVICE}

Again, if the script is tested and working you should create a Magisk module for it.

How to install Magisk into the boot partition using a script

URL: https://xdaforums.com/t/how-to-install-magisk-into-the-boot-partition-using-a-script.4456621/

How to install Magisk into the boot partition using a script

Update 13.04.2023 /bs

To install Magisk v26.0 or newer the script version 25.01.2023 v2.5.1.0 or newer is necessary. Unfortunately there is a new feature in Magisk v26.x that prevents the full installation of Magisk from within a booted recovery; from the docs:

This means that any installation of Magisk v26+ using custom recovery will be incomplete; a subsequent re-installation through the Magisk app after booting up is required.

Therefor the initial installation of Magisk v26.x via script without user intervention does not work but the update of an installed Magisk v25.x to Magisk v26.x via script works.
So to install Magisk v26.x via script install Magisk v25.x and then update to Magisk v26.x.

I wrote a little shell script for Linux (bash) to add Magisk to the boot partition of a phone without user intervention called install_magisk_via_twrp.sh.

This is useful for example to re-install Magisk into the boot partition after an OS upgrade .
The script uses only the commands fastboot and adb, a recovery image (e.g. TWRP), and the patch script from Magisk to re-install Magisk into the boot partition.

Notes

Note that the script version 1.1.0.0 or newer also works if the phone is secured via PIN or something similar.
Since version 2.0.0.0 it is not necessary anymore to install Magisk via Magisk App into a boot image file before starting the script.
Since version 2.0.0.0 the script can install Magisk into the partition even if the Magisk app is not installed yet.
SInce version 2.1.0.0 the script also copys the Magisk apk file to the phone if requested via parameter.
The version 2.5.1.0 (or newer) of the script is necessary to install Magisk v26.1 or newer
Since version 4.0.0.0 of the script the include file boot_phone_from_twrp.sh is required

The usage for the script is:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./install_magisk_via_twrp.sh -h install_magisk_via_twrp.sh version - v4.0.0.0 - add Magisk to the boot partition of a phone running Android using TWRP install_magisk_via_twrp.sh [-h|help|-H] [boot_slot] [wait=n] [--reboot|--noreboot] [dd|fastboot] [twrp_image] [cleanup] [delete_adb_dir] [adb_only] [use_apk] [copy_apk] [magisk_apk_file=file] [oldmethod|newmethod] The script boot_phone_from_twrp.sh is required by this script -- see the source code of the script [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Use the parameter -H to print the detailed usage help:

Detailed usage

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./install_magisk_via_twrp.sh -H install_magisk_via_twrp.sh version - v4.0.0.0 - add Magisk to the boot partition of a phone running Android using TWRP Note: Since version 4.0.0.0 this script needs the helper script boot_phone_from_twrp.sh Usage install_magisk_via_twrp.sh [-h|help|-H] [boot_slot] [wait=n] [--reboot|--noreboot] [dd|fastboot] [twrp_image] [cleanup] [delete_adb_dir] [adb_only] [use_apk] [copy_apk] [magisk_apk_file=file] [oldmethod|newmethod] The script boot_phone_from_twrp.sh is required by this script -- see the source code of the script All parameter are optional. The parameter can be used in any order. Use the parameter "help" or "-H" to print the detailed usage help; use the parameter "-h" to print only the short usage help The parameter "boot_slot" can be a, b, active, inactive, next, current; default is the current boot slot of the phone The value for the parameter "wait" is the number of seconds to wait before starting the script to install Magisk on the phone This seems to be necessary to avoid errors while repacking the boot image. The default wait time is 10 seconds. Use the parameter "dd" to request repatching via dd in an adb session ; use the parameter "fastboot" to request repatching via "fastboot" Default is to use "dd" to flash the patched boot image. The parameter "twrp_image" can be used to define another TWRP image to use. The parameter is optional - the default for "twrp_image" is hardcoded in the script (variable TWRP_IMAGE). The default TWRP image of the script is the TWRP for the ASUS Zenfone 8. If the parameter "delete_adb_dir" is used the script will delete all files and directories created in the directory /data after Magisk was successfully installed into the boot partition If the parameter "adb_only" is used the script will only install the directories and binaries for Magisk in the directory /data/adb. The script will unpack the necesseray files for adding Magisk to the boot partition from the installed Magisk apk file to the temporary directory /data/local/tmp/MagiskInst if the files in /data/adb/magisk are missing. Use the parameter "use_apk" to force the script to use the files from the Magisk app even if the files in /data/adb/magisk exist. Use the parameter "cleanup" to delete the temporary directory with Magisk at script end If the parameter "copy_apk" is used the script will copy the Magisk apk file to the phone. Use the parameter "magisk_apk_file=file" to install Magisk from another apk file The script checks the contents of the Magisk apk file to detect if the Magisk version is v26.0 or newer. To overwrite the result of this check use the parameter "old_method" (= assume the Magisk version is v25.x or older) or "new_method" (= assume the Magisk version is v26.x or newer) Note that installing Magisk via the new method only works if adb is already enabled in the installed Android OS. Note further that this method is only possible if the parameter "copy_apk" and "use_apk" are also used. Therefor the parameter "new_method" will also enable the parameter "copy_apk" and "use_apk". The script now can also install Magisk into a boot partition without an installed Magisk app. To use this feature copy the Magisk apk file (Magisk*apk) to the directory /data or /sdcard/Download on the phone before starting the script or use the parameter "copy_apk". The phone to patch must be attached via USB. The phone can be either in fastboot mode, in normal mode with enabled adb support, or already booted from the TWRP image To change some of the values used by the script you can set environment variables before starting the script: Set the environment variable REBOOT to "yes" before starting the script to automatically reboot the phone after patching the new image Set the environment variable REBOOT to "no" before starting the script to disable the automatic reboot after patching the new image. Set the environment variable SERIAL_NUMBER to the serial number of the phone to patch if there is more then one phone connected via USB Set the environment variable ADB_OPTIONS to the options to be used with the adb command Set the environment variable FASTBOOT_OPTIONS to the options to be used with the fastboot command Set the environment variable DOWNLOAD_DIR_ON_PHONE to the data directory to use on the phone (def.: /sdcard/Download) Set the envionment variable MAGISK_APK_FILE to the Magisk apk file to use Set the environment variable FASTBOOT_WAIT_TIME to change the time out value to wait for booting into the fastboot mode Set the environment variable ADB_BOOT_WAIT_TIME to change the time out value to wait until adb access is working after a reboot Set the environment variable DECRYPT_DATA_WAIT_TIME to change the time out value to wait until the partition /data is decrypted after a reboot The script uses the script boot_phone_from_twrp.sh The default TWRP image to use is "/data/backup/ASUS_ZENFONE8/twrp/current_twrp.img" [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

In the default the script will ask the user for input before rebooting the phone from the patched boot partition.

To disable this request and automatically reboot the phone call the script like this

REBOOT=yes install_magisk_via_twrp.sh

REBOOT=no install_magisk_via_twrp.sh

to disable the user request and do not reboot the phone.

Additional infos

Prerequisites

the Magisk App; either installed or as apk file
the script boot_phone_from_twrp.sh
a computer running Linux with working adb and fastboot binaries available via PATH variable
a phone with unlocked boot loader
a working recovery image (e.g. TWRP) for the attached phone that automatically mounts /data and has enabled adb support (For the ASUS Zenfone 8 with the original Android from ASUS or the OmniROM the official TWRP image from https://twrp.me/asus/zenfone8.html version 3.7.0 or newer can be used)

Test Environment

Tested on an ASUS Zenfone 8 and with

OmniROM 12 (Android 12) and Magisk v24.3
OmniROM 12 (Android 12) and Magisk v25.0
OmniROM 12 (Android 12) and Magisk v25.2
OmniROM 13 (Android 13) and Magisk v25.2
OmniROM 14 (Android 14) and Magisk v25.2, v26.x, and v27.x
AospExtended 9.0 w/o GAPPS (Android 12) and Magisk v24.3
AospExtended 9.0 w/o GAPPS (Android 12) and Magisk v25.0
ASUS Original Android 11 and Magisk v24.3
ASUS Original Android 12 and Magisk v24.3
ASUS Original Android 13 Beta 1 and Magisk v25.0 and Magisk v27.x

StatixOS (Android 14) and Magisk v27.x
LineageOS 20 (Android 13) and Magisk v26.x
LineageOS 21 (Android 14) and Magisk v27.x
LMODroid (Android 14) and Magisk v27.x

Details

The script uses the scripts and binaries from the directory /data/adb/magisk. If these executables do not exist the script searches for the installed Magisk apk file /data/app/*/com.topjohnwu.magisk*/base.apk and extracts the scripts and binaries from that file. If the Magisk apk is not installed the script searches for the Magisk APK file Magisk*.apk in the directories /sdcard/Download and /data and extracts the scripts and executables from that file if found.
If the parameter copy_apk is used the script copies the Magisk apk file to the directory /sdcard/Download on the phone or, if that directory does not exist, to the directory /data on the phone
The patched boot images are created in the directory /sdcard/Downloads on the phone (variable DOWNLOAD_DIR_ON_PHONE)
The script uses the original script called boot_patch.sh from Magisk to patch the boot image.
You can execute the script as often as you like.
The script does not delete the created files in the directory /sdcard/Download on the phone so you might do a cleanup of that directory manual
If the directory /sdcard/Download is not available the script uses the directory /tmp for temporary files (Note: /tmp is mounted on a ramdisk)
If the check sum of the boot image before the patching is equal to the check sum after the successfull patching then Magisk was already installed in the boot partition
The script boots the phone up to 4 times.

Trouble Shooting

- Error message like this

 /data/adb/magisk/boot_patch.sh[211]: can't create
        /proc/self/fd/: Is a directory

can be ignored (these are only error messages from the function to print some additional messages)

Note: Since the script version 2.0.0.0 these error messages should not occure anymore.

- Patching the boot partition via "dd" fails sometimes for unknown reason so you should only use it if you know how to fix a damaged boot partition!

- Downloading the patched boot image via "adb pull" (neccessary for patching the boot partition using fastboot) sometimes fails for unknown reason:

So if you get an error message like this:

 [ 77%] /sdcard/Download/patched_boot_a.391985.img ERROR:
        Error downloading the file
        "/sdcard/Download/patched_boot_a.391985.img" from the phone!

just restart the script again (or download the file and flash the boot partition manually)

- If the adb connection dies and there are error messages like this

 

        adb: insufficient permissions for device

restart the adb server using
adb kill-server

if that does not work disconnect and reconnect the USB cable

Example output of the script install_magisk_via_twrp.sh

[xtrnaw7@t15g /data/backup/Android/scripts_on_linux]$
          ./install_magisk_via_twrp.sh

          ./install_magisk_via_twrp.sh version - 2.0.0.0 - add Magisk to
          the boot partition of a phone running Android using TWRP

          

          Using the TWRP image hardcoded in the script:
          "/data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D.img" 

          Checking the script prerequisites ...

          Will wait 10 second(s) before starting the script on the phone

          The script is running as user "xtrnaw7" -- will use "sudo" for
          the fastboot commands ...

          Will patch the boot partition on the attached phone with the
          serial number "M6AIB760D0939LX"

          Using the options "-d -s M6AIB760D0939LX " for the adb
          commands 

          Using the options " -s M6AIB760D0939LX " for the fastboot
          commands 

          Checking for a connected phone booted into fastboot mode ...

          No attached phone in fastboot mode found

          Checking for an attached phone with working access via adb
          (USB) ...

          Linux localhost 5.4.147-Omni-qgki-perf-gea6f815f5af9-dirty #42
          SMP PREEMPT Tue Sep 27 18:03:41 CEST 2022 aarch64

          ... found a phone connected via USB with working adb access

          The phone is booted in normal mode

          The installed OS is based on Android 12 (Wed Aug 10 06:59:46
          CEST 2022); the description for the distribution is
          "omni_zenfone8-user 12 SQ3A.220605.009.A1
          eng.xtrnaw.20220810.065947 release-keys" 

          The installed version of Magisk is com.topjohnwu.magisk
          versionCode 25200 

          Booting the phone into the fastboot mode now ...

          Waiting up to 60 seconds for the boot into the fastboot mode
          ...

          Booting the phone from the TWRP image
          "/data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D.img"
          now ...

          Sending 'boot.img' (98304 KB) OKAY [ 2.489s]

          Booting OKAY [ 10.359s]

          Finished. Total time: 12.888s

          Waiting up to 60 seconds for the boot of the phone ....

          Retrieving the current boot slot from the phone ...

          The current boot slot is "_b" 

          The boot slot to patch is "_b" 

          The boot partition to patch is "boot_b" 

          Checking if the Magisk patch script "boot_patch.sh" exists on
          the phone ....

          -rwxr-xr-x 1 root root 5987 2022-10-27 07:33
          /data/adb/magisk/boot_patch.sh

          Checking if the download directory "/sdcard/Download" exists
          on the phone ....

          /sdcard/Download

          Creating the boot image file
          "/sdcard/Download/boot_b.732627.img" from the partition
          "/dev/block/by-name/boot_b" ...

          196608+0 records in

          196608+0 records out

          100663296 bytes (96 M) copied, 0.308243 s, 311 M/s

          

          Checking the result ...

          -rw-rw---- 1 root media_rw 100663296 2022-10-28 09:51
          /sdcard/Download/boot_b.732627.img

          The check sums are:

          The check sum of the boot partition
          "/dev/block/by-name/boot_b" on the phone is "4108888341" 

          The check sum of th boot image file on the phone is
          "/sdcard/Download/boot_b.732627.img" is "4108888341" 

          Patching the boot image file
          "/sdcard/Download/boot_b.732627.img" ...

          Waiting now 10 seconds ...

          - Unpacking boot image

          Parsing boot image: [/sdcard/Download/boot_b.732627.img]

          HEADER_VER [3]

          KERNEL_SZ [42025472]

          RAMDISK_SZ [34080597]

          OS_VERSION [12.0.0]

          OS_PATCH_LEVEL [2022-05]

          PAGESIZE [4096]

          CMDLINE []

          KERNEL_FMT [raw]

          RAMDISK_FMT [gzip]

          VBMETA

          - Checking ramdisk status

          Loading cpio: [ramdisk.cpio]

          - Stock boot image detected

          - Patching ramdisk

          Loading cpio: [ramdisk.cpio]

          Add entry [init] (0750)

          Create directory [overlay.d] (0750)

          Create directory [overlay.d/sbin] (0750)

          Add entry [overlay.d/sbin/magisk64.xz] (0644)

          Patch with flag KEEPVERITY=[false] KEEPFORCEENCRYPT=[false]

          Found fstab file [system/etc/fstab.postinstall]

          Remove pattern
          [,avb_keys=/product/etc/security/avb/system_other.avbpubkey]

          Loading cpio: [ramdisk.cpio.orig]

          Backup mismatch entry: [init] -> [.backup/init]

          Record new entry: [overlay.d] -> [.backup/.rmlist]

          Record new entry: [overlay.d/sbin] -> [.backup/.rmlist]

          Record new entry: [overlay.d/sbin/magisk64.xz] ->
          [.backup/.rmlist]

          Backup mismatch entry: [system/etc/fstab.postinstall] ->
          [.backup/system/etc/fstab.postinstall]

          Create directory [.backup] (0000)

          Add entry [.backup/.magisk] (0000)

          Dump cpio: [ramdisk.cpio]

          - Repacking boot image

          Parsing boot image: [/sdcard/Download/boot_b.732627.img]

          HEADER_VER [3]

          KERNEL_SZ [42025472]

          RAMDISK_SZ [34080597]

          OS_VERSION [12.0.0]

          OS_PATCH_LEVEL [2022-05]

          PAGESIZE [4096]

          CMDLINE []

          KERNEL_FMT [raw]

          RAMDISK_FMT [gzip]

          VBMETA

          Repack to boot image: [new-boot.img]

          HEADER_VER [3]

          KERNEL_SZ [42025472]

          RAMDISK_SZ [34331854]

          OS_VERSION [12.0.0]

          OS_PATCH_LEVEL [2022-05]

          PAGESIZE [4096]

          CMDLINE []

          Checking the result ...

          -rw-r--r-- 1 root root 100663296 2022-10-28 09:51
          /data/adb/magisk/new-boot.img

          The patched boot image is
          "/sdcard/Download/patched_boot_b.732627.img" 

          Waiting now 5 seconds ...

          Patching the partition "/dev/block/by-name/boot_b" from the
          patched boot image file
          "/sdcard/Download/patched_boot_b.732627.img" via dd ...

          196608+0 records in

          196608+0 records out

          100663296 bytes (96 M) copied, 4.721236 s, 20 M/s

          Checking the result ....

          

          The check sums for the images and devices on the phone are:

          

          4108888341 100663296 /sdcard/Download/boot_b.732627.img

          3369959638 100663296
          /sdcard/Download/patched_boot_b.732627.img

          3369959638 100663296 /dev/block/by-name/boot_b

          

          OK, patching the boot partition "/dev/block/by-name/boot_b"
          was successfull

          

          *** Press return to reboot the phone now

          

          Rebooting the phone now ...

          [xtrnaw7@t15g /data/backup/Android/scripts_on_linux]$

History of this entry

Update 03.07.2022 /bs

The script now uses the directory /tmp for temporary files if the directory /sdcard/Download is not available. Therefor it will also work if the phone is secured by a PIN or something similar.

Update 28.10.2022 /bs

Updated the post to match the new script version 2.0.0.0

Update 17.12.2022 /bs

Updated the post to match the new script version 2.1.0.0

Update 30.12.2022 /bs

Updated the post to match the new script version 2.3.0.0

Update 13.04.2023 /bs

To install Magisk v26.0 or newer the script version 25.01.2023 v2.5.1.0 is necessary. Unfortunately there is a new feature in Magisk v26.x that prevents the full installation of Magisk from within a booted recovery; from the docs:

This means that any installation of Magisk v26+ using custom recovery will be incomplete; a subsequent re-installation through the Magisk app after booting up is required.

Therefor currently the installation of Magisk v26.x via script without user intervention does not yet work

Update 14.07.2024 /bs

Updated the details to match the current version of the scripts and tests

Update 18.07.2024 /bs

Some of the links in this section did not work -- fixed

Update 07.08.2024 /bs

Fixed the link for the script /files/public/Android/boot_phone_from_twrp.sh (there was a leading "." in the URL)

How to change files in the directory /system with Magisk

URL: https://xdaforums.com/t/how-to-change-files-in-the-directory-system-with-magisk.4454489/

How to change files in the directory /system with Magisk

How to change files in the directory /system with Magisk

Note:

I tested the instructions below with Magisk 24.3, 25.0, and 25.2 on an ASUS Zenfone 8 running OmniROM (Android 12).

The filesystem for /system is normally mounted read-only in Android 12 and it's difficult to change files in that filesystem.

One possible solution to change files in /system is to create a Magisk module for this task.

For testing if a Magisk Module would work like expected you can create a dummy Magisk module. The steps to implement the Dummy Magisk Module are:

Install Magisk, enable root access for the shell in Magisk, and enable access via adb (this can be done in the "Developer Options" in the system settings)

Next connect to the phone using the command

adb shell

and become root user

su -

Now create a sub directory in the directory

/data/adb/modules

e.g.

ASUS_I006D:/ # ls -ld /data/adb/modules/* 

        drwxrws--- 3 u0_a118 media_rw 3452 2022-05-30 12:12
        /data/adb/modules/playstore 

        ASUS_I006D:/ #

Then copy the new files for the directory /system to that directory (/data/adb/modules/<modulename> is equal here to / on the phone)

Example:

ASUS_I006D:/ # find /data/adb/modules/playstore 

        /data/adb/modules/playstore 

        /data/adb/modules/playstore/system 

        /data/adb/modules/playstore/system/etc 

        /data/adb/modules/playstore/system/etc/permissions 

        /data/adb/modules/playstore/system/etc/permissions/com.android.vending.xml



















        

        /data/adb/modules/playstore/system/priv-app 

        /data/adb/modules/playstore/system/priv-app/Phonesky 

        /data/adb/modules/playstore/system/priv-app/Phonesky/lib 

        /data/adb/modules/playstore/system/priv-app/Phonesky/lib/arm 

        /data/adb/modules/playstore/system/priv-app/Phonesky/lib/arm/libbrotli.so



















        

        /data/adb/modules/playstore/system/priv-app/Phonesky/lib/arm/libconscrypt_jni.so



















        

        /data/adb/modules/playstore/system/priv-app/Phonesky/lib/arm/libcronet.85.0.4181.5.so



















        

        /data/adb/modules/playstore/system/priv-app/Phonesky/lib/arm/libgame_sdk_device_info_jni.so



















        

        /data/adb/modules/playstore/system/priv-app/Phonesky/lib/arm/libphonesky_data_loader.so



















        

        /data/adb/modules/playstore/system/priv-app/Phonesky/lib/arm/libtensorflowlite_jni.so



















        

        /data/adb/modules/playstore/system/priv-app/Phonesky/Phonesky.apk



















        

        /data/adb/modules/playstore/system/priv-app/FakeStore 

        /data/adb/modules/playstore/system/priv-app/FakeStore/oat 

        /data/adb/modules/playstore/system/priv-app/FakeStore/oat/arm64
        

        /data/adb/modules/playstore/system/priv-app/FakeStore/oat/arm64/FakeStore.odex



















        

        /data/adb/modules/playstore/system/priv-app/FakeStore/oat/arm64/FakeStore.vdex



















        

        /data/adb/modules/playstore/system/priv-app/FakeStore/FakeStore.apk



















        

        ASUS_I006D:/ #

Now reboot the phone.

After the reboot all files in /data/adb/modules/<modulename>/system will be mapped to /system by Magisk, e.g:

ASUS_I006D:/ # find /system/priv-app/Phonesky/ 

        /system/priv-app/Phonesky/ 

        /system/priv-app/Phonesky/lib 

        /system/priv-app/Phonesky/lib/arm 

        /system/priv-app/Phonesky/lib/arm/libbrotli.so 

        /system/priv-app/Phonesky/lib/arm/libconscrypt_jni.so 

        /system/priv-app/Phonesky/lib/arm/libcronet.85.0.4181.5.so 

        /system/priv-app/Phonesky/lib/arm/libgame_sdk_device_info_jni.so
        

        /system/priv-app/Phonesky/lib/arm/libphonesky_data_loader.so 

        /system/priv-app/Phonesky/lib/arm/libtensorflowlite_jni.so 

        /system/priv-app/Phonesky/Phonesky.apk 

        

        ASUS_I006D:/ # ASUS_I006D:/ # find /system/priv-app/FakeStore/ 

        /system/priv-app/FakeStore/ 

        /system/priv-app/FakeStore/oat 

        /system/priv-app/FakeStore/oat/arm64 

        /system/priv-app/FakeStore/oat/arm64/FakeStore.vdex 

        /system/priv-app/FakeStore/oat/arm64/FakeStore.odex 

        /system/priv-app/FakeStore/FakeStore.apk 

        ASUS_I006D:/ #

Since the mapping is performed when the phone boots, it will also survive the update of the operating system on the phone to a new version as long as the user data is not deleted and Magisk is installed.

Notes:

Magisk "replaces" existing directories and files in /system with the files in this directory by using bind mounts.

You can not add additional files or directories to /system using this method -- it's only possible to add additional files and directories to the existing sub directories in /system.

This method to replace files in /system simulates a module in Magisk so if everything is okay I strongly recommend to create a "real" Magisk Module with the new files for /system

This Magisk feature can also be used to make files in sub directories in /system writable -- see How to make files in system writable for detailed instructions.

This method can also be used to replace files in the directories /vendor, /product, or /system_ext:

from https://topjohnwu.github.io/Magisk/guides.html:

The system folder

All files you want to replace/inject should be placed in this folder. This folder will be recursively merged into the real /system; that is: existing files in the real /system will be replaced by the one in the modules system, and new files in the modules system will be added to the real /system.

If you place a file named .replace in any of the folders, instead of merging its contents, that folder will directly replace the one in the real system. This can be very handy for swapping out an entire folder.

If you want to replace files in /vendor, /product, or /system_ext, please place them under system/vendor, system/product, and system/system_ext respectively. Magisk will transparently handle whether these partitions are in a separate partition or not

But this seems to work only to replace existing files in the directories /vendor, /product, or /system_ext.

See How to change any file or directory using Magisk for another approach to change files on read-only mounted filesystems.

Update 14.08.2024

This method also works with Magisk 26.x and 27.x on OS based on Android 13 or Android 14.

Trouble Shooting

If something does not work check the Magisk log file in /cache :

/cache/magisk.log

History of this entry

History

Update 10.06.2022

added the infos about how to update files /vendor, /product, /system-ext

Update 12.06.2022

The Magisk Module to install the patched Playstore used in this example can be downloaded here

https://bnsmb.de/files/public/Android/PlayStore_for_MicroG.zip

see also here

Update 03.10.2022

The sentence about how to update files in /vendor, /product, /system-ext was missing here --fixed
Added a note about what can not be done using this method

Update 04.11.2022

Added the hint for the post to make files in /system writable
Doing some cleanup to make the instructions more clear

Update 18.01.2023

Added a link to the new post How to change any file or directory using Magisk

How to backup the data from the phone using rsync and ssh (including some hints for using sshd on an Android phone)

URL: https://xdaforums.com/t/how-to-backup-the-data-from-the-phone-using-rsync-and-ssh-including-some-hints-for-using-sshd-on-an-android-phone.4462327/

How to backup the data from the phone using rsync and ssh (including some hints for using sshd on an Android phone)

How to backup the data from the phone using rsync and ssh (including some hints for using sshd on an Android phone)

Like for all computer it's important to have a backup of the data on the phone.

For those who like me don't like to save their private data in one of the suspicious clouds there is a solution with standard Linux tools:

Use rsync and ssh to backup the data from the phone to your local workstation (see the man page for rsync for details regarding rsync and why it is useful for this task)

The neccessary tools for Android for this method can be installed with the Magisk Module MagiskSSH.

Download the Magisk Module with MagiskSSH from here

https://gitlab.com/d4rcm4rc/MagiskSSH_releases

Copy the ZIP file with the Magisk Module to the phone :

adb push magisk_ssh_v0.14.zip /sdcard/Download/

and install it via the module installation from within the Magisk App or manuell using :

adb shell su - -c /data/adb/magisk/magisk64 --install-module
        /sdcard/Download/magisk_ssh_v0.14.zip

Sample output of the installation

ASUS_I006D:/ # /data/adb/magisk/magisk64 --install-module
          /sdcard/Download/magisk_ssh_v0.14.zip

           - Current boot slot: _a

           - Device is system-as-root 

          ******************************* OpenSSH for Android
          ******************************* 

          [0/7] Preparing module directory 

          [1/7] Extracting architecture unspecific module files 

          [2/7] Extracting libraries and binaries for arm64 

          [3/7] Configuring library path wrapper 

          [4/7] Recreating symlinks 

          [5/7] Creating SSH user directories 

          [6/7] Found sshd_config, will not copy a default one 

          [7/7] Cleaning up - Setting permissions - Done 

          ASUS_I006D:/ #

A reboot is required now.

adb shell reboot

For the next tasks open a adb shell and become root user.

Next create the authorized_keys file for the user root :

touch /data/ssh/root/.ssh/authorized_keys 

        chmod 600 /data/ssh/root/.ssh/authorized_keys

and add your public ssh key to the file /data/ssh/root/.ssh/authorized_keys.

Update 29.06.2024

Depending on the config of the sshd the permissions for the directories /data/ssh/root and /data/ssh/root/.ssh directory must be more restricted:

chmod 750 /data/ssh/root chmod 600 /data/ssh/root/.ssh

To make sure that the keys and other data files for the MagiskSSH module are not removed while deinstalling the module you should create the file /data/ssh/KEEP_ON_UNINSTALL:

touch /data/ssh/KEEP_ON_UNINSTALL

The MagiskSSH module also installs a service to start sshd after each reboot: to disable this start create the file /data/ssh/no-autostart:

touch /data/ssh/no-autostart

To manually start or stop the sshd use the script /data/adb/modules/ssh/opensshd.init :

# start the sshd (as user root) 

        # 

        /data/adb/modules/ssh/opensshd.init start 

        

        # to stop the sshd (as user root) 

        # 

        /data/adb/modules/ssh/opensshd.init stop

Now test the access via ssh from your Linux workstation:
ssh -l root <phone_ip_address> id
To retrieve the IP address of the phone this command can be used:

PHONE_IP_ADDRESS=$( adb shell ifconfig wlan0 | grep "inet
        addr:" | sed -e "s/.*inet addr://g" -e
        "s/[[:space:]]*Bcast.*//g" )

example :

[xtrnaw7@t15g ~]$ ssh -l root ${PHONE_IP_ADDRESS} id 

        uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 

        [xtrnaw7@t15g ~]$

Now you can use rsync to backup the data from the phone, e.g. to backup the photos from the phone do :

# on your local Linux workstation do: 

        # start the sshd on the phone via adb if not already running 

        # 

        adb shell su - -c /data/adb/modules/ssh/opensshd.init start 

        

        # retrieve the current IP address from the phone 

        # 

        PHONE_IP_ADDRESS=$( adb shell ifconfig wlan0 | grep "inet addr:"
        | sed -e "s/.*inet addr://g" -e "s/[[:space:]]*Bcast.*//g" ) 

        

        # backup the new photos from the phone to the Linux workstation
        (rsync only copies new files from the phone) 

        # to the local directory /data/backup/ASUS_ZENFONE8/DCIM 

        # 

        rsync -av --rsync-path /data/adb/modules/ssh/usr/bin/rsync
        root@${PHONE_IP_ADDRESS}:/sdcard/DCIM/
        /data/backup/ASUS_ZENFONE8/DCIM 

        

        # optional stop the sshd on the phone via adb 

        # 

        adb shell su - -c /data/adb/modules/ssh/opensshd.init stop

Note: The sshd configuration file used is /data/ssh/sshd_config

Troubleshooting

Update 29.06.2024

If the ssh access does not work, open an adb session and start the sshd in debug mode as user root:

/data/adb/modules/ssh/usr/bin/sshd -d

and check the messages of the sshd.

(In debug mode the sshd stops after the ssh session ends)

Sample script to backup all data in the directory /sdcard

#!/bin/bash # # simple script to backup the data of an phone using adb, ssh, and rsync # # History # 27.06.2022 /bs # initial release # # for testing # #RSYNC_OPTIONS="${RSYNC_OPTIONS} --dry-run" RSYNC_OPTIONS="${RSYNC_OPTIONS} --del " # default is to backup the phone connected via adb over LAN # [ $# -ne 0 ] && ADB_OPTIONS="$*" || ADB_OPTIONS="-e" # retrieve the serial number of the attached phone # SERIAL_NO="$( adb ${ADB_OPTIONS} shell getprop ro.serialno )" if [ "${SERIAL_NO}"x = ""x ] ; then echo "ERROR: Can not read the serial number of the connected phone" exit 89 fi VENDOR_MODEL="$( adb ${ADB_OPTIONS} shell getprop ro.product.vendor.model )" BACKUP_DIR="/data/backup/ASUS_ZENFONE8/data_backup/${VENDOR_MODEL}_${SERIAL_NO}" if [ ! -d "${BACKUP_DIR}" ] ; then echo "ERROR: The directory \"${BACKUP_DIR}\" does not exist" exit 99 fi PHONE_IP_ADDRESS="$( adb ${ADB_OPTIONS} shell ifconfig wlan0 | grep "inet addr:" | sed -e "s/.*inet addr://g" -e "s/[[:space:]]*Bcast.*//g" )" if [ "${PHONE_IP_ADDRESS}"x = ""x ] ; then echo "ERROR: Can not detect the IP address of the phone" exit 100 fi echo "Updating a backup of the data on the phone with the serial number \"${SERIAL_NO}\" and the IP \"${PHONE_IP_ADDRESS}\" to the directory \"${BACKUP_DIR}\" ..." set -x # start the sshd if neccessary # adb ${ADB_OPTIONS} shell su - -c /data/adb/modules/ssh/opensshd.init start # do the backup # time rsync ${RSYNC_OPTIONS} -av --rsync-path /data/adb/modules/ssh/usr/bin/rsync root@${PHONE_IP_ADDRESS}:/sdcard/ "${BACKUP_DIR}/" # stop the sshd # adb ${ADB_OPTIONS} shell su - -c /data/adb/modules/ssh/opensshd.init stop set +x

How to enable access via ssh for non-root user

How to enable access via ssh for non-root user

In the standard configuration installed by MagiskSSH ssh access is only allowed as user root because the ssh keys are in the directory /data and all non-root user can not read files in the directory /data. Therefor some efforts are neccessary to add ssh access for non-root user.

e.g. To enable the ssh access for the user shell do:

To configure ssh access for the user shell we must create a .ssh directory for the user shell in a directory tree owned by the user shell. The only directory on the phone owned by the user shell that can be used for this purpose is /storage :

ASUS_I006D:/ # ls -ld /storag

        drwx--x--- 4 shell everybody 80 2022-06-26 18:37 /storage 

        ASUS_I006D:/ #

But unfortunately all files and directories in this directory are temporary and will be deleted after a reboot of the phone.

Therefor we configure a startup script in Magisk to create this directory tree after each reboot, e.g.

/data/adb/service.d/create_ssh_dir_for_shell.sh:

# /data/adb/service.d/create_ssh_dir_for_shell.sh

        # 

        mkdir -p /storage/shell/.ssh chmod -R 700 /storage/shell/ 

        touch /storage/shell/.ssh/authorized_keys 

        echo "<ssh_public_key>" >
        /storage/shell/.ssh/authorized_keys 

        chmod 600 /storage/shell/.ssh/authorized_keys 

        chown -R shell:shell /storage/shell

To test the script just execute it one time manually as user root.

su - -c sh data/adb/service.d/create_ssh_dir_for_shell.sh

Now create a backup of the sshd config file

su - -c cp /data/ssh/sshd_config
        /storage/ssh/sshd_config.org.$$

and add these lines at the end of the file /data/ssh/sshd_config

Match User shell AuthorizedKeysFile
        /storage/shell/.ssh/authorized_keys

Restart the sshd if it's already running

Now test the access as user shell, example:

[xtrnaw7@t15g ~]$ ssh -l shell 192.168.1.148 id 

        uid=2000(shell) gid=2000(shell) groups=2000(shell)
        context=u:r:magisk:s0 

        [xtrnaw7@t15g ~]$

The reason for this config is the setting "StrictMode yes" in the sshd config file /data/ssh/sshd_config (see the man page for sshd_config for details). So another "solution" is to change this setting:

With the setting "StrictModes no" in the file sshd_config the directory with the authorized_keys file for the non-root users can be anywhere (for example in /sdcard/shell)

Execute as user root:

sed -i -e "s/.*StrictModes.*//g" -e
        "s/UsePrivilegeSeparation/StrictModes
        no\nUsePrivilegeSeparation/g" /data/ssh/sshd_config

and change the entry in the file /data/ssh/sshd_config for the authorized_keys file for the user shell, for example:

Match User shell AuthorizedKeysFile
        /sdcard/shell/.ssh/authorized_keys

Afterwards restart the sshd:

/data/adb/modules/ssh/opensshd.init stop 

        /data/adb/modules/ssh/opensshd.init start

Now create the directories and files neccessary for the ssh access (see above) in the directory /sdcard/shell:

ASUS_I006D:/ # find /sdcard/shell -exec ls -ld {} \; 

        drwxrws--- 3 u0_a118 media_rw 3452 2022-06-26 18:32
        /sdcard/shell 

        drwxrws--- 2 u0_a118 media_rw 3452 2022-06-26 18:32
        /sdcard/shell/.ssh 

        -rw-rw---- 1 u0_a118 media_rw 408 2022-06-26 18:32
        /sdcard/shell/.ssh/authorized_keys 

        ASUS_I006D:/ #

and the access as user shell via ssh should work

Update 14.07.2024

The sub directory /data/local/tmp can be accessed by the user shell. Therefor the .ssh directory for the user shell can be /data/local/tmp/.ssh. This directory and it's contents survive a reboot and therefor the Magisk init script is not neccessary.

How to install packages (apk files) for Android via script

URL: https://xdaforums.com/t/how-to-install-packages-apk-files-for-android-via-script.4464987/

How to install packages (apk files) for Android via script

Sometimes, for example after a reinstallation of the phone from scratch, it's quite useful to install all the apps via a script.

The Android command that can be used to install an app in a script is:

pm install [apk_file]

But unfortunately that does not work, neither as user shell (due to missing file access rights) nor as user root:

ASUS_I006D:/ # id uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 ASUS_I006D:/ # ASUS_I006D:/ # pm install /sdcard/Download/EssentialApps/MiXplorer_v6.58.4-API29_B22020920.apk avc: denied { read } for scontext=u:r:system_server:s0 tcontext=u:object_r:fuse:s0 tclass=file permissive=0 System server has no access to read file context u:object_r:fuse:s0 (from path /sdcard/Download/EssentialApps/MiXplorer_v6.58.4-API29_B22020920.apk, context u:r:system_server:s0) Error: Unable to open file: /sdcard/Download/EssentialApps/MiXplorer_v6.58.4-API29_B22020920.apk Consider using a file under /data/local/tmp/ Error: Can't open file: /sdcard/Download/EssentialApps/MiXplorer_v6.58.4-API29_B22020920.apk Exception occurred while executing 'install': java.lang.IllegalArgumentException: Error: Can't open file: /sdcard/Download/EssentialApps/MiXplorer_v6.58.4-API29_B22020920.apk at com.android.server.pm.PackageManagerShellCommand.setParamsSize(PackageManagerShellCommand.java:572) at com.android.server.pm.PackageManagerShellCommand.doRunInstall(PackageManagerShellCommand.java:1337) at com.android.server.pm.PackageManagerShellCommand.runInstall(PackageManagerShellCommand.java:1303) at com.android.server.pm.PackageManagerShellCommand.onCommand(PackageManagerShellCommand.java:193) at com.android.modules.utils.BasicShellCommandHandler.exec(BasicShellCommandHandler.java:97) at android.os.ShellCommand.exec(ShellCommand.java:38) at com.android.server.pm.PackageManagerService.onShellCommand(PackageManagerService.java:24812) at android.os.Binder.shellCommand(Binder.java:950) at android.os.Binder.onTransact(Binder.java:834) at android.content.pm.IPackageManager$Stub.onTransact(IPackageManager.java:4818) at com.android.server.pm.PackageManagerService.onTransact(PackageManagerService.java:8887) at android.os.Binder.execTransactInternal(Binder.java:1184) at android.os.Binder.execTransact(Binder.java:1143) 255|ASUS_I006D:/ #

After a short search on the Internet, I found a workaround for that issue on this page:

https://stackoverflow.com/questions/50540334/install-apk-using-root-handling-new-limitations-of-data-local-tmp-folder

pm install also supports the installation of a package piped to STDIN, e.g

cat [package_file] | pm install -S [package_size]

Example:

127|ASUS_I006D:/sdcard/Download # ls -l OmniStore.apk -rw-rw---- 1 u0_a120 media_rw 4601419 2022-07-03 16:37 OmniStore.apk ASUS_I006D:/sdcard/Download # ASUS_I006D:/sdcard/Download # cat OmniStore.apk | pm install -S 4601419 Success ASUS_I006D:/sdcard/Download #

Or, on the PC:

[xtrnaw7@t15g /data/backup/Android]$ adb shell id uid=2000(shell) gid=2000(shell) groups=2000(shell),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),1078(ext_data_rw),1079(ext_obb_rw),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats),3009(readproc),3011(uhid) context=u:r:shell:s0 [xtrnaw7@t15g /data/backup/Android]$ [xtrnaw7@t15g /data/backup/Android]$ cat EssentialApps/F-Droid.apk | adb shell pm install -S $( ls -l EssentialApps/F-Droid.apk | awk '{ print $5 }' ) Success [xtrnaw7@t15g /data/backup/Android]$

So I wrote a little script to install one or more packages on an Android phone using this method.

The script can run on a PC with a connected phone with a working adb connection or in a shell on the phone

The usage of the script is:

# # Usage on a phone: # # install_apk.sh [apk1|dir1 ... apk#|dir#] # # Usage on a PC: # # install_apk.sh [options_for_adb --] [apk1|dir1 ... apk#|dir#] # # apk# is the name of an apk file to install; dir# is a directory with apk files # If a parameter is a directory the script will install all files with the extension .apk from that directory # # The options for adb are optional; the script will not check the options for adb # the number of apk files or directories is only limited by the maxium parameter supported by the used shell. # # Set the variable PM_INSTALL_OPTIONS with additional options for the "pm install" command before starting the # script if neccessary # # Prerequisites # The packages to install must exist as file either on the PC or on the phone # A shell on the phone or via adb command is required # root access is NOT neccessary #

Sample script output for running on a phone:

ASUS_I006D:/storage/emulated/0/Download $ sh ./install_apk.sh myapps/ OmniStore.apk Running on a phone Directory found in the parameter: Installing all apk files found in the directory "myapps/" The apks will be installed on the phone model ASUS_I006D with the serial number M6AIB760D0939LX Installing these apks myapps//Magisk-25.1_25100.apk myapps//Magisk-v25.0.apk myapps//MiXplorer_v6.58.4-API29_B22020920.apk myapps//Notecase_Pro_1.2.2.apk myapps//com.keramidas.TitaniumBackupAddon_v1.apk myapps//com.keramidas.TitaniumBackup_v417.apk OmniStore.apk Installing the apk "myapps//Magisk-25.1_25100.apk" ... Success "myapps//Magisk-25.1_25100.apk" succcessfully installed Installing the apk "myapps//Magisk-v25.0.apk" ... Failure [INSTALL_FAILED_VERSION_DOWNGRADE: Package Verification Result] ERROR: Error installing the apk "myapps//Magisk-v25.0.apk" Installing the apk "myapps//MiXplorer_v6.58.4-API29_B22020920.apk" ... Success "myapps//MiXplorer_v6.58.4-API29_B22020920.apk" succcessfully installed Installing the apk "myapps//Notecase_Pro_1.2.2.apk" ... Success "myapps//Notecase_Pro_1.2.2.apk" succcessfully installed Installing the apk "myapps//com.keramidas.TitaniumBackupAddon_v1.apk" ... Success "myapps//com.keramidas.TitaniumBackupAddon_v1.apk" succcessfully installed Installing the apk "myapps//com.keramidas.TitaniumBackup_v417.apk" ... Success "myapps//com.keramidas.TitaniumBackup_v417.apk" succcessfully installed Installing the apk "OmniStore.apk" ... Success "OmniStore.apk" succcessfully installed Installation summary ==================== 6 package(s) successfully installed: myapps//Magisk-25.1_25100.apk myapps//MiXplorer_v6.58.4-API29_B22020920.apk myapps//Notecase_Pro_1.2.2.apk myapps//com.keramidas.TitaniumBackupAddon_v1.apk myapps//com.keramidas.TitaniumBackup_v417.apk OmniStore.apk 1 package(s) not installed: myapps//Magisk-v25.0.apk ASUS_I006D:/storage/emulated/0/Download $

Sample script output for running on a PC:

[xtrnaw7@t15g /data/backup/Android/myapps]$ /data/develop/android/install_apk.sh -d -- /data/backup/Android/myapps/ ./Notecase/Notecase_Pro_1.2.1.apk Running on a PC Using adb with the options " -d " to install the packages Directory found in the parameter: Installing all apk files found in the directory "/data/backup/Android/myapps/" The apks will be installed on the phone model ASUS_I006D with the serial number M6AIB760D0939LX Installing these apks /data/backup/Android/myapps//com.android.keepass_196.apk /data/backup/Android/myapps//com.keramidas.TitaniumBackup_v417.apk /data/backup/Android/myapps//com.matoski.adbm_v27.apk /data/backup/Android/myapps//F-Droid.apk /data/backup/Android/myapps//FoxMagiskModuleManager_0.4.0-rc1.apk /data/backup/Android/myapps//Magisk-25.1_25100.apk /data/backup/Android/myapps//MiXplorer_v6.58.4-API29_B22020920.apk /data/backup/Android/myapps//Notecase_Pro_1.2.2.apk ./Notecase/Notecase_Pro_1.2.1.apk Installing the apk "/data/backup/Android/myapps//com.android.keepass_196.apk" ... Success "/data/backup/Android/myapps//com.android.keepass_196.apk" succcessfully installed Installing the apk "/data/backup/Android/myapps//com.keramidas.TitaniumBackup_v417.apk" ... Success "/data/backup/Android/myapps//com.keramidas.TitaniumBackup_v417.apk" succcessfully installed Installing the apk "/data/backup/Android/myapps//com.matoski.adbm_v27.apk" ... Success "/data/backup/Android/myapps//com.matoski.adbm_v27.apk" succcessfully installed Installing the apk "/data/backup/Android/myapps//F-Droid.apk" ... Success "/data/backup/Android/myapps//F-Droid.apk" succcessfully installed Installing the apk "/data/backup/Android/myapps//FoxMagiskModuleManager_0.4.0-rc1.apk" ... Success "/data/backup/Android/myapps//FoxMagiskModuleManager_0.4.0-rc1.apk" succcessfully installed Installing the apk "/data/backup/Android/myapps//Magisk-25.1_25100.apk" ... Success "/data/backup/Android/myapps//Magisk-25.1_25100.apk" succcessfully installed Installing the apk "/data/backup/Android/myapps//MiXplorer_v6.58.4-API29_B22020920.apk" ... Success "/data/backup/Android/myapps//MiXplorer_v6.58.4-API29_B22020920.apk" succcessfully installed Installing the apk "/data/backup/Android/myapps//Notecase_Pro_1.2.2.apk" ... Success "/data/backup/Android/myapps//Notecase_Pro_1.2.2.apk" succcessfully installed ERROR: The file "./Notecase/Notecase_Pro_1.2.1.apk" does not exist or is not readable Installation summary ==================== 8 package(s) successfully installed: /data/backup/Android/myapps//com.android.keepass_196.apk /data/backup/Android/myapps//com.keramidas.TitaniumBackup_v417.apk /data/backup/Android/myapps//com.matoski.adbm_v27.apk /data/backup/Android/myapps//F-Droid.apk /data/backup/Android/myapps//FoxMagiskModuleManager_0.4.0-rc1.apk /data/backup/Android/myapps//Magisk-25.1_25100.apk /data/backup/Android/myapps//MiXplorer_v6.58.4-API29_B22020920.apk /data/backup/Android/myapps//Notecase_Pro_1.2.2.apk 1 package(s) not found: ./Notecase/Notecase_Pro_1.2.1.apk [xtrnaw7@t15g /data/backup/Android/myapps]$ adb push /data/develop/android/install_apk.sh /sdcard/Download/ /data/develop/android/install_apk.sh: 1 file pushed, 0 skipped. 76.3 MB/s (5906 bytes in 0.000s) [xtrnaw7@t15g /data/backup/Android/myapps]$

Download the script from here install_apk.sh

Notes:

Another method to install packages via script from an PC is

adb install [package]

This command copies the file to a temporary location on the phone and installs it then via "pm install".

e.g

[xtrnaw7@t15g /data/backup/Android/myapps]$ adb install F-Droid.apk Performing Streamed Install Success [xtrnaw7@t15g /data/backup/Android/myapps]$

Another ugly and insecure method on the phone to install an apk file using pm install as root is

pm install (insecure method)

130|ASUS_I006D:/ # id uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 ASUS_I006D:/ # ASUS_I006D:/ # getenforce Enforcing ASUS_I006D:/ # setenforce 0 ASUS_I006D:/ # ASUS_I006D:/ # getenforce Permissive ASUS_I006D:/ # ASUS_I006D:/ # pm install /sdcard/Download/EssentialApps/com.matoski.adbm_v27.apk avc: denied { read } for scontext=u:r:system_server:s0 tcontext=u:object_r:fuse:s0 tclass=file permissive=1 Success ASUS_I006D:/ # setenforce 1 ASUS_I006D:/ # ASUS_I006D:/ # getenforce Enforcing ASUS_I006D:/ #

pm install supports a lot of options -- to get the usage help for pm use

adb shell pm help

or on the phone

pm help

or see here: https://developer.android.com/studio/command-line/adb#pm (Google Account neccessary) or here: http://adbcommand.com/adbshell/pm

Update 18.12.2023

Android 14 does not allow the installation of apks based on outdated SDK versions anymore. Trying to install these kind of apks ends with an error message like this:

ASUS_I006D:/ $ ls -l /sdcard/Download/com.matoski.adbm_v27.apk -rw-rw---- 1 u0_a125 media_rw 793506 2022-03-08 22:24 /sdcard/Download/com.matoski.adbm_v27.apk ASUS_I006D:/ $ cat /sdcard/Download/com.matoski.adbm_v27.apk | pm install -S 793506 Failure [INSTALL_FAILED_DEPRECATED_SDK_VERSION: App package must target at least SDK version 23, but found 19] 1|ASUS_I006D:/ $
To get around this error use the additional parameter --bypass-low-target-sdk-block for the command pm install, e.g.:

ASUS_I006D:/ $ ls -l /sdcard/Download/com.matoski.adbm_v27.apk -rw-rw---- 1 u0_a125 media_rw 793506 2022-03-08 22:24 /sdcard/Download/com.matoski.adbm_v27.apk 1|ASUS_I006D:/ $ 1|ASUS_I006D:/ $ cat /sdcard/Download/com.matoski.adbm_v27.apk | pm install --bypass-low-target-sdk-block -S 793506 Success ASUS_I006D:/ $

To use this parameter with the script install_apk.sh define the environment variable PM_INSTALL_OPTIONS before executing the script:

[ OmniRom 13 Dev - xtrnaw7@t15g ~ ] $ export PM_INSTALL_OPTIONS="--bypass-low-target-sdk-block " [ OmniRom 13 Dev - xtrnaw7@t15g ~ ] $ /data/develop/android/scripts/install_apk.sh /data/backup/Android/EssentialApps/com.matoski.adbm_v27.apk Running on a PC Using adb to install the packages The apks will be installed on the phone model ASUS_I006D with the serial number M6AIB760D0939LX Installing these apks /data/backup/Android/EssentialApps/com.matoski.adbm_v27.apk The additional parameter for the command "pm install" are: "--bypass-low-target-sdk-block " Installing the apk "/data/backup/Android/EssentialApps/com.matoski.adbm_v27.apk" ... Success "/data/backup/Android/EssentialApps/com.matoski.adbm_v27.apk" succcessfully installed Installation summary ==================== 1 package(s) successfully installed: /data/backup/Android/EssentialApps/com.matoski.adbm_v27.apk [ OmniRom 13 Dev - xtrnaw7@t15g ~ ] $

Note:

Due to the outdated SDK used to create the app, it is important to test wether the app really works on Android 14!

Update 02.05.2024

If the installation of an apk file using the command pm install -S <apk_size> fails with an error message like this:

Failure [INSTALL_PARSE_FAILED_NOT_APK: Failed to parse /data/app/vmdl643615185.tmp/base.apk: Failed to load asset path /data/app/vmdl643615185.tmp/base.apk]
most probably an invalid value for the size of the package to install was used.

There was an error in the script install_apk.sh that used the wrong file size for installing apks using symbolic links - this error is fixed in the script version from the 02.05.2024.

Update 04.08.2024

The script install_apk.sh now supports parameter to define the options for the command "pm install":

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./install_apk.sh -h Usage: install_apk.sh [options_for_adb -- ] [options_for_pm_install] [--old] [apk1|dir1 ... apk#|dir#] Use the parameter "-h -v" to print the detailed usage help [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Detailed usage help

[OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./install_apk.sh -h -v Usage: install_apk.sh [options_for_adb -- ] [options_for_pm_install] [--old] [apk1|dir1 ... apk#|dir#] Use the parameter "-h -v" to print the detailed usage help "options_for_adb" are the options for the command "adb". The parameter "--" is mandatory if adb options are specified in the parameter. The options for adb are optional; the script does not check the options for adb. options for adb are not allowed if running a shell on the phone. "options_for_pm" are the options for the command "pm install". the script does not check the options for the "pm install" command. "--old" is a short cut for the "pm install" command option "--bypass-low-target-sdk-block"; this option is necessary to install apks build for outdated sdk versions. apk# is the name of an apk file to install; dir# is a directory with apk files If a parameter is a directory the script installs all files with the extension .apk from that directory The number of apk files or directories is only limited by the maxium parameter supported by the used shell. Additional parameter for the "pm install" command can also defined in the environment variable PM_INSTALL_OPTIONS before starting the script. Usefull options for "pm install": --bypass-low-target-sdk-block -> install apk file with outdated sdk version -g -> apply all runtime permissions -r -d -> downgrade an apk (this only works if the pm command is executed as root user) examples: su - -c 'PM_INSTALL_OPTIONS="-r -d " /data/local/tmp/install_apk.sh /sdcard/Download/com.machiav3lli.backup_8316.apk ' or su - -c "/data/local/tmp/install_apk.sh -r -d /sdcard/Download/com.machiav3lli.backup_8316.apk " To use adb via WLAN enable adb via WiFi on the phone, start the adb for WLAN on the PC, example: adb -e -L tcp:localhost:5237 connect 192.168.1.148:6666 and then use a command like this to install apks using the script: ./install_apk.sh -e -L tcp:localhost:5237 -- /data/Downloads/com.machiav3lli.backup_8317.apk [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Notes

The parameter for the "pm install" command to downgrade a package are "-r -d" . Downgrading a package via "pm install" command in Android 14 is only allowed if the command is executed by the user root.

To use the script to downgrade a package execute the script on the phone like this:

su - -c "/data/local/tmp/install_apk.sh -r -d /sdcard/Download/com.machiav3lli.backup_8316.apk

To manually downgrade a package using the GUI on the phone the App SimpleAppDowngrader can be used:

https://f-droid.org/en/packages/com.garyodernichts.downgrader/

Note that the app needs root access to downgrade an app

How to replace the Fake Store from OmniROM with MicroG with a patched Playstore

URL: https://xdaforums.com/t/magisk-module-to-replace-the-fakestore-in-the-omnirom-with-a-patched-playstore-from-nanodroid.4456421/

How to replace the Fake Store from OmniROM with MicroG with a patched Playstore

The OmniROM with MicroG contains a perfect integrated MicroG but it does not contain the Playstore. Therefor apps that rely on the License functionality of the Playstore (like for example the DWD Warnwetter App) work only partial.

Fortunately there is already a patched PlayStore that works with MicroG in the NanoDroid image.

Therefor I created a Magisk Module to replace the FakeStore in OmniROM with MicroG with the patched PlayStore from NanoDroid.

The Module will replace the apk for the FakeStore from MicroG with an empty file and add the directory

/system/priv-apps/Phonesky

with the patched PlayStore from NanoDroid.

The patched Playstore works with MicroG so that apps using the license functionality from the Playstore can also be used in ROMs using MicroG .

The ZIP file with the Magisk Module can be downloaded from here:

https://bnsmb.de/files/public/Android/PlayStore_for_MicroG.zip

Note:

Update 09.07.2023

MicroG and the Playstore used in NanoDroid are now outdated and do not work anymore on the newest Android version . Therefor I recommend to use the Magisk Module microG-GApps to install MicroG and a working Playstore on CustomROMs without the Google stack.

see also How to install a more recent version of the Playstore in OmniROM

How to change the home directory for the user root on an Android phone

URL: https://xdaforums.com/t/how-to-change-the-home-directory-for-the-user-root-on-an-android-phone.4469305/

How to change the home directory for the user root on an Android phone

When working a lot in a shell via adb on an Android phone it's usefull to be able to store some user dependent config files in the home directory of the user. So let's see how that can be implemented on a phone running Android.

Note: The config was done on OmniROM (based on Android 12)

The home directory for all user used in an (adb) shell on Android is the root directory "/".

Using a home directory for more then one user is not really usefull and in addition on Android "/" is mounted read-only so more or less useless as home directory.

Therefor this should be changed. Unfortunately there is no /etc/passwd file in Android to configure the home directory for a user.

Well, there is an /etc/passwd file (probably for compatibily reasons) but it's empty:

shell@ASUS_I006D:/ $ ls -l /etc/passwd -rw-r--r-- 1 root root 0 2009-01-01 01:00 /etc/passwd shell@ASUS_I006D:/ $

And as far as I know there is no other config file to configure the home directory of the user in the Android OS.

So we must implement some work around to get this working.

The shell on Android behaves like normal shells on Linux and executes the file /etc/profile when starting a new session.

In Android /etc is a symbolic link to /system/etc:

root@ASUS_I006D:/ # ls -ld /etc lrw-r--r-- 1 root root 11 2009-01-01 01:00 /etc -> /system/etc root@ASUS_I006D:/ #

And in the default config there is no file called profile in that directory:

shell@ASUS_I006D:/ $ ls -l /etc/system/profile ls: /etc/system/profile: No such file or directory shell@ASUS_I006D:/ $

Because /system is also mounted read-only we need the magic tool Magisk again to create the file /system/etc/profile.

To create the file /system/etc/profile create these directories and files on the phone as user root (assuming Magisk is already installed on the phone).

root@ASUS_I006D:/ # find /data/adb/modules/initshell/ /data/adb/modules/initshell/ /data/adb/modules/initshell/system /data/adb/modules/initshell/system/etc /data/adb/modules/initshell/system/etc/profile root@ASUS_I006D:/ #

and reboot the phone. After the reboot there should be the writable file /etc/profile:

root@ASUS_I006D:/ # ls -l /etc/profile -rw-rw-rw- 1 root root 1100 2022-07-14 14:51 /etc/profile root@ASUS_I006D:/ #

which is in reality the file

/data/adb/modules/initshell/system/etc/profile

Now you can edit the file /etc/profile as user root until it fullfills your requirements (be aware that all changes in that file are now persistent).
To test the changes open a new adb session (you should not close the current adb session to be able to fix an error in the profile if opening a new adb session fails).

This file can now be used to define the home directory for the user. The home directory should be on one of the writable filesystem, e.g. in /data:

# add in /etc/profile HOME="/data/home/root" export HOME

and the result is:

root@ASUS_I006D:/ # id uid=0(root) gid=0(root) groups=0(root),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),1078(ext_data_rw),1079(ext_obb_rw),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats),3009(readproc),3011(uhid) context=u:r:su:s0 root@ASUS_I006D:/ # echo $HOME /data/home/root root@ASUS_I006D:/ #

Note that there is not really a writable filesystem for the user shell so this approach to change the home directory is mainly usable for the user root.

The file /etc/profile can also be used to init some other settings for (adb) sessions (e.g. change the PATH, etc) . This feature can be used for sessions for non-root users also.

For additional user dependent configs you can also create the file .profile in the home directory of the user; the file ${HOME}/.profile will be executed after the execution of the file /etc/profile.

Notes:

Be aware that creating a profile for the user root and changing the home directory for the user root might have some side effects on other processes using the shell!

Therefor it's recommended to use the command "tty -s" to test if the profile is executed in an interactive session:

# # check if we're running in an interactive session # if ! tty -s; then # # this is not an interactive session - so we're just doing nothing at all # : else # # running in an interactive session # ... fi
You can also check if the parent process is the adb daemon to not do anything in non-adb sessions:

ps -fp $PPID| grep adbd >/dev/null if [ $? -ne 0 ] ; then
        # # not running in an adb session : else # # running in an adb
        session ... fi

For testing purpose I suggest to open at least two shells via adb to be able to fix an error in case opening a new shell fails due to a bug in the profile.

If opening a shell session fails due to an error in the /etc/profile and you do not have another open shell either delete the file /data/adb/modules/initshell/system/etc/profile with a filemanager with root access on the phone or boot the phone from a recovery image (like TWRP) and delete or edit the file /data/adb/modules/initshell/system/etc/profile.

I initially looked at this feature to enable a persistent command history for the shell on the Android phone. But after some google searches I found out that persistent history shells are disabled for the shell binary on Android. So this will not work without recompiling the shell binary.

The attached example for the profile uses the directory /data/home/root as home directory for the user root and does not change the home directory for other users. The profile does nothing if not running in an interactive session.
The directory /data/home/root is created by the profile if it does not yet exist.

As always, if the config is working you should create a real Magisk Module for the profile.

The Magisk Module attached, initshell.zip, can be used to create a Magisk Module with your own profile:

To create a new Magisk Module with your own /etc/profile do:

# create an empty working directory # TEMPDIR="/tmp/newdir" mkdir "${TEMPDIR}" cd "${TEMPDIR}" # unpack the zip file in the new directory # unzip ../initshell.zip # now edit the file ${TEMPDIR}/system/etc/profile # also (optional) edit the files config.sh and module.prop in the new directory to document your changes # the script customize.sh from the module will be executed once when the module is installed. You might add the code # create the home directories or any other code here # # and recreate the zip file # zip -r ../initshell.zip .

Some hints for using Magisk on Android phones

URL: https://xdaforums.com/t/some-hints-for-using-magisk-on-android-phones.4471857/

Some hints for using Magisk on Android phones

In this post I summarize everything I found out about using Magisk (https://topjohnwu.github.io/Magisk) until now. A lot of the infos in this post are already in my other posts in this board but hopefully this summary is useful anyway.

My main purpose is to configure Magisk as much as possible via scripts so this tutorial is mostly about using Magisk via CLI commands in an adb shell or from within scripts.

I will update the post whenever I found something new and interesting. A history of changes in this post is at the end of the post

PreRequisites

Most of the instructions below need an enabled adb access for the phone and root permissions for the adb shell. In addition, a working Recovery image that is able to mount the /data partition (like for example TWRP) is strongly recommended before starting to play with Magisk.

The Magisk versions used for this tutorial are Magisk 25.1 and Magisk 25.2; the ROM used was OmniROM (Android 12 based).
The phone used was an ASUS Zenfone 8 but the instructions should work on other phones also.

Installation

Magisk consists out of two parts:

The Magisk App and the Magisk files in the boot partition. Without the files in the boot partition Magisk can do more or less nothing.

The Magisk App can be installed using the standard Android tools for installing apps.

The installation of the Magisk files into the boot partition can then be done using the Magisk App.

The official instructions to install Magisk into the boot partition are here:

https://topjohnwu.github.io/Magisk/install.html

Magisk can also be installed via script without user intervention into the boot partition -- see here: How to install Magisk into the boot partition using a script

Update 18.04.2023/bs

see How to install Magisk v26.0 or newer via script for how to install Magisk v26 via script

Note that the Magisk files for the boot partition must be reinstalled after an OS upgrade.

Uninstalling Magisk

Use the Button "Uninstall Magisk" in the Magisk app. The Magisk App can also remove Magisk from the boot partition.

To uninstall the Magisk App only via script do

adb shell pm uninstall com.topjohnwu.magisk

To remove Magisk from the boot partition manually I suggest to flash the image of the original boot partition.

Installing Magisk Modules

Magisk Modules are extensions for Magisk to for example replace or add files to /system (and much more ... I suggest to study the documentation for Magisk for those that are interested in using Magisk)

Magisk Modules are distributed via ZIP files.

As of Magisk v25 or newer the official method to install Magisk Modules is to manually copy the ZIP file with the Magisk Module to the phone and then use the button "Install from storage" in the "Modules" sub menu in the Magisk App.

In previous versions of Magisk a Module browser to list and install Magisk Modules was part of the Magisk App but unfortunately the developer dropped that feature. Therefor you must search and download the available Magisk Modules using a browser on the phone or a PC.

Repositories with Magisk Modules are for example:

https://github.com/Magisk-Modules-Repo/

and

https://github.com/Magisk-Modules-Alt-Repo

A XDA thread about Magisk Modules (including a list of Magisk Modules) is here:

https://xdaforums.com/t/collection-of-magisk-modules-v2.3575758/

There is now a new App for Android called "Fox's Magisk Module Manager" that can be used to list and download Magisk Modules to the phone
(see https://github.com/Fox2Code/FoxMagiskModuleManager)

It's also possible to install a Magisk Module via script. The commands to install an Magisk Module via script are:

# copy the Magisk Module to the phone adb push /data/backup/Android/MagiskModules/Nano_for_Android_NDK-6_3-6300.zip /sdcard/Download/ # install the Module # # Note: # # The command can also be executed in a script runnning on the phone (without "adb shell") # adb shell magisk --install-module /sdcard/Download/Nano_for_Android_NDK-6_3-6300.zip # reboot the phone adb reboot

Example:

[xtrnaw7@t15g ~]$ adb push /data/backup/Android/MagiskModules/Nano_for_Android_NDK-6_3-6300.zip /sdcard/Download/ /data/backup/Android/MagiskModules/Nano_for_Android_NDK-6_3-6300.zip: 1 file pushed, 0 skipped. 37.6 MB/s (380554 bytes in 0.010s) [xtrnaw7@t15g ~]$ adb shell magisk --install-module /sdcard/Download/Nano_for_Android_NDK-6_3-6300.zip - Current boot slot: _a - Device is system-as-root ***************************** Nano for Android NDK by osm0sis @ xda-developers ***************************** ******************* Powered by Magisk ******************* Archive: /storage/emulated/0/Download/Nano_for_Android_NDK-6_3-6300.zip inflating: diffusion_config.sh inflating: module.prop Mounting... Extracting files... Archive: /storage/emulated/0/Download/Nano_for_Android_NDK-6_3-6300.zip inflating: META-INF/com/google/android/updater-script inflating: META-INF/com/google/android/update-binary inflating: sbin/nano ... inflating: etc/terminfo/l/linux inflating: customize.sh inflating: update.json inflating: bin/nano inflating: bin/nano.bin inflating: README.md inflating: diffusion_config.sh inflating: module.prop Installing... Installing nano to /data/adb/modules_update/nano-ndk/system/bin ... Installing terminfo to /data/adb/modules_update/nano-ndk/system/etc ... Unmounting... Done! [xtrnaw7@t15g ~]$ adb reboot [xtrnaw7@t15g ~]$

Uninstalling a Magisk Module

Magisk Modules should be uninstalled via Magisk App if possible because there is no parameter for the magisk binary to uninstall a Magisk Module.

To force Magisk to remove the module after the next reboot create the file

remove

in the module directory.

If uninstalling the Magisk Module does not work or if you want to uninstall a Magisk Module via script you can just delete the directory for the Magisk Module in /data/adb/modules and reboot the phone. This can be done in an adb shell or if that does not work anymore after rebooting the phone from a Recovery Image like TWRP, e.g.

# uninstall the module "dummy_module" via adb shell commands # # check for an uninstall script for this module if [ -x /data/adb/modules/dummy_module/uninstall.sh ] ; then echo "Executing the module uninstall script ...." /data/adb/modules/dummy_module/uninstall.sh fi adb shell rm -rf /data/adb/modules/dummy_module/ adb reboot

Note:

You should reboot the phone immediately after removing the directory with the Module. This method is not really recommended.

List all installed Magisk Modules

The Magisk App lists all installed Magisk Modules

To list the installed Magisk Modules via CLI command use either

adb shell su - -c ls -d /data/adb/modules/* | cut -f5 -d "/"

e.g.

[xtrnaw7@t15g ~]$ adb shell su - -c ls -d /data/adb/modules/* | cut -f5 -d "/" PlayStore_for_MicroG ccbins dummy_module initshell nano-ndk terminalmods [xtrnaw7@t15g ~]$

or

[xtrnaw7@t15g /data/develop/android/NewModules/initshell]$ for i in $( adb shell su - -c ls -d /data/adb/modules/* ); do printf "%-30s %s\n" "${i##*/}" "$( adb shell su - -c grep name= $i/module.prop 2>/dev/null | cut -f2 -d "=" | tr -s " " )" ; done MiXplorer                      MiXplorer PlayStore_for_MicroG           Patched Playstore from NanoDroid for MicroG (for ARM64 CPUs only) ccbins                         Cross Compiled Binaries dummy_module                   Dummy Module for testing a new module initshell                      Create writable config files (/etc/profile) for sh nano-ndk                       Nano for Android NDK terminalmods                   Terminal Modifications [xtrnaw7@t15g /data/develop/android/NewModules/initshell]$

or on the phone:

root@ASUS_I006D:/ # echo; for i in /data/adb/modules/*; do printf "%-30s %s\n" "${i##*/}" "$( grep name= $i/module.prop 2>/dev/null | cut -f2 -d "=" | tr -s " " )" ; done        PlayStore_for_MicroG           Patched Playstore from NanoDroid for MicroG (for ARM64 CPUs only) ccbins                         Cross Compiled Binaries dummy_module                   initshell                      Create writable config files (/etc/profile) for sh nano-ndk                       Nano for Android NDK terminalmods                   Terminal Modifications root@ASUS_I006D:/ #

How to disable a Magisk Module

To disable a Magisk Module via script create a file called disable in the Module directory, e.g.: touch /data/adb/modules/dummy_module/disable The Magisk Module will then be disabled after the next reboot. To re-enable the Magisk Module just delete the file disable in the Module directory and reboot the phone. To list all disabled Magisk Modules do adb shell ls /data/adb/modules/*/disable | cut -f5 -d "/"

Replacing files in /system using a Dummy Magisk Module

One feature of Magisk Modules is the possibility to change files in the sub directories in the directory /system or also add new files to the sub directories in the directory /system. /system is mounted read-only and in the current Android version it is not possible to remount /system read-write anymore so without Magisk it's very difficult to change files in /system.
Please note that you can not create new files or directories in the directory /system using this Magisk feature.

To test this feature you do not need to create a Magisk Module : It's sufficient to simulate a Magisk Module.
For that just create the necessary directory structure and files manually.

Magisk Modules are are installed in the directory /data/adb/modules. Each Magisk Module use an uniqe sub directory in that directory. To simulate a Magisk Module open an adb shell as user root and create the directories for your Dummy Magisk Module, e.g.:

mkdir /data/adb/modules/dummy_module # Next create the sub directory system in that new directory mkdir /data/adb/modules/dummy_module/system

Now copy the files for /system to that directory: The files in that directory will overwrite existing files in /system after the next reboot. Files in that directory that not already exist in /system will be created in /system.

e.g

# contents of my Dummy Magisk Module # root@ASUS_I006D:/data/adb/modules/dummy_module/system # pwd /data/adb/modules/dummy_module/system root@ASUS_I006D:/data/adb/modules/dummy_module/system # find . . ./bin ./bin/my_new_binary ./etc ./etc/my_new_file_for_systme root@ASUS_I006D:/data/adb/modules/dummy_module/system #

After rebooting the phone the files are visible in /system:

root@ASUS_I006D:/ # ls -l /system/etc/my_new_file_for_systme -rw-rw-rw- 1 root root 20 2022-07-20 16:55 /system/etc/my_new_file_for_systme root@ASUS_I006D:/ # cat /system/etc/my_new_file_for_systme This is my new file root@ASUS_I006D:/ # root@ASUS_I006D:/ # ls -l /system/bin/my_new_binary -rwxr-xr-x 1 root root 41 2022-07-20 16:57 /system/bin/my_new_binary root@ASUS_I006D:/ # my_new_binary Hello world from my dummy binary root@ASUS_I006D:/ #

The permissions for the new files in /system are equal to the permissions of the original files in /data/adb/modules/dummy_module so if the original files are writable you can also edit the new files in /system, e.g

root@ASUS_I006D:/ # cat /system/etc/my_new_file_for_systme This is my new file root@ASUS_I006D:/ # root@ASUS_I006D:/ # echo "Test Test" >> /system/etc/my_new_file_for_systme root@ASUS_I006D:/ # cat /system/etc/my_new_file_for_systme                                                                                                                    This is my new file Test Test root@ASUS_I006D:/ #

These changes are persistent (because they change the original file in /data/adb/modules/<modulename>). You can also edit the files in /data/adb/modules/<modulename> but be aware that you can not use tools that create temporary files for changing the files like for example sed: Changes done with these kind of tools are only visible after a reboot of the phone.

To change the file via script I recommend to create a new file and then replace the existing file with the new file using cp.

This Magisk feature can also be used to make files in sub directories in /system writable -- see How to make files in system writable for detailed instructions.

Notes:

Magisk does not really change the files in /system - instead it uses "bind mounts" to replace the files with others. Therefor you can always restore the original files by deleting the files in /data/adb/modules/<module_name> and rebooting the phone. Another big advantage of this technique is that you do not have to take care about an update of the OS:

Changes done via Magisk will also work after an OS upgrade (assuming you reinstalled Magisk to the boot partition after installing the OS update)

Make sure that the new files for /system are readable for the user using them; e.g some files (like for example apk files) are used by the user system). IMHO it's recommended to add the read permission (chmod o+r) to all new files for /system.

Please be careful when changing existing files in /system.

To add the necessary infos for the Magisk App to list your dummy module properly just create the file module.prop for your Dummy Magisk Module, e.g.

root@ASUS_I006D:/ # cat /data/adb/modules/dummy_module/module.prop id=dummy-module name=Dummy Module for testing a new module version=1.0 versionCode=1000 author=otto.meier@mynetwork.com description=Dummy Module .. root@ASUS_I006D:/ #

For some examples for this technique see :

How to change files in the directory /system with Magisk

How to replace the Fake Store from OmniROM with MicroG with a patched Playstore

How to change the home directory for the user root on an Android phone

How to use this technique to disable an app that can not be uninstalled

Apps that can not be uninstalled can be disabled by creating empty files for the app.

Example:

My Magisk Module for the patched Playstore must disable the installed app for the fake store

That's done by creating an empty file in the Magisk Module for the apk file, the result looks like this:

root@ASUS_I006D:/ # ls -l /data/adb/modules/PlayStore_for_MicroG/system/priv-app/FakeStore/FakeStore.apk -rw-r--r-- 1 root root 1 2022-07-15 11:54 /data/adb/modules/PlayStore_for_MicroG/system/priv-app/FakeStore/FakeStore.apk root@ASUS_I006D:/ # root@ASUS_I006D:/ # ls -l /system/priv-app/FakeStore/FakeStore.apk -rw-r--r-- 1 root root 1 2022-07-15 11:54 /system/priv-app/FakeStore/FakeStore.apk root@ASUS_I006D:/ # In case there are multiple files in the folder for the app you can also hide the entire folder:From the original documentation from https://topjohnwu.github.io/Magisk/guides.html:

If you place a file named .replace in any of the folders, instead of merging its contents, that folder will directly replace the one in the real system. This can be very handy for swapping out an entire folder.

How to access the files replaced by Magisk

Use another bind mount to access the files replaced by Magisk.

Example:

root@ASUS_I006D:/ # head /data/adb/modules/fmradio/system/etc/public.libraries.txt# See https://android.googlesource.com/platform/ndk/+/master/docs/PlatformApis.md # # 04.08.2022 /bs # added the library libqcomfm_jni.so # libandroid.so libaaudio.so libamidi.so libbinder_ndk.so libc.so root@ASUS_I006D:/ # root@ASUS_I006D:/ # root@ASUS_I006D:/ # head /system/etc/public.libraries.txt# See https://android.googlesource.com/platform/ndk/+/master/docs/PlatformApis.md # # 04.08.2022 /bs # added the library libqcomfm_jni.so # libandroid.so libaaudio.so libamidi.so libbinder_ndk.so libc.so root@ASUS_I006D:/ # # -> The file /system/etc/public.libraries.txt is replaced by a file from a Magisk Module # To access the original version of the file /system/etc/public.libraries.txt do root@ASUS_I006D:/ # root@ASUS_I006D:/ # mkdir -p /data/test root@ASUS_I006D:/ # root@ASUS_I006D:/ # mount -o bind /system /data/test ## -> /data/test is now "bind" mounted to /system ## -> /data/test/etc/public.libraries.txt is the original version of that file: # root@ASUS_I006D:/ # root@ASUS_I006D:/ # head /data/test/etc/public.libraries.txt # See https://android.googlesource.com/platform/ndk/+/master/docs/PlatformApis.md libandroid.so libaaudio.so libamidi.so libbinder_ndk.so libc.so libcamera2ndk.so libdl.so libEGL.so libGLESv1_CM.so root@ASUS_I006D:/ #

Replacing files in /vendor, /product, or /system_ext using a Dummy Magisk Module

The (dummy) Magisk Module can also be used to replace files in the directories /vendor, /product, or /system_ext:

from the Magisk documentation at https://topjohnwu.github.io/Magisk/guides.html:

If you want to replace files in /vendor, /product, or /system_ext, please place them under system/vendor, system/product, and system/system_ext respectively. Magisk will transparently handle whether these partitions are in a separate partition or not.

Note that this works only to replace existing files in these directories - you can not add new files to these directories using this method.

Example:

root@ASUS_I006D:/ # cd /data/adb/modules/dummy_module/system # # files in the Magisk Module # root@ASUS_I006D:/data/adb/modules/dummy_module/system # find . -type f ./bin/my_new_binary ./etc/my_new_file_for_systme ./product/mynewfile ./product/media/audio/alarms/Krypton.ogg ./vendor/mynewfile ./vendor/etc/sap.conf ./system_ext/mynewfile ./system_ext/etc/dpm/dpm.conf root@ASUS_I006D:/data/adb/modules/dummy_module/system # # # the new files for /product, /vendor, and /system_ext in the Magisk Module do not exist: # root@ASUS_I006D:/data/adb/modules/dummy_module/system # ls -l /product/mynewfile /vendor/mynewfile /system_ext/mynewfile ls: /product/mynewfile: No such file or directory ls: /vendor/mynewfile: No such file or directory ls: /system_ext/mynewfile: No such file or directory # # The existing files in /product, /vendor, and /system_ext are replaced with the files from the Magisk Moduel # root@ASUS_I006D:/data/adb/modules/dummy_module/system # cksum ./product/media/audio/alarms/Krypton.ogg /product/media/audio/alarms/Krypton.ogg 4294967295 0 ./product/media/audio/alarms/Krypton.ogg 4294967295 0 /product/media/audio/alarms/Krypton.ogg root@ASUS_I006D:/data/adb/modules/dummy_module/system # root@ASUS_I006D:/data/adb/modules/dummy_module/system # cksum ./vendor/etc/sap.conf /vendor/etc/sap.conf 1967598015 7121 ./vendor/etc/sap.conf 1967598015 7121 /vendor/etc/sap.conf root@ASUS_I006D:/data/adb/modules/dummy_module/system # root@ASUS_I006D:/data/adb/modules/dummy_module/system # cksum ./system_ext/etc/dpm/dpm.conf /system_ext/etc/dpm/dpm.conf 1970692378 2925 ./system_ext/etc/dpm/dpm.conf 1970692378 2925 /system_ext/etc/dpm/dpm.conf root@ASUS_I006D:/data/adb/modules/dummy_module/system # root@ASUS_I006D:/data/adb/modules/dummy_module/system # grep mynewfile /cache/magisk.log 01-01 00:11:51.764 767 769 W : Unable to add: /product/mynewfile, skipped 01-01 00:11:51.771 767 769 W : Unable to add: /system_ext/mynewfile, skipped 01-01 00:11:51.771 767 769 W : Unable to add: /vendor/mynewfile, skipped root@ASUS_I006D:/data/adb/modules/dummy_module/system #

A work around to create new files in sub directories in /product, /vendor, or /system_ext:

Copy the complete existing folder to the sub directory in /data/adb/modules/<magisk_module>/system/[product|vendor|system_ext]/; create the file .replace in that directory and add the new file(s) to that directory.

Example:

root@ASUS_I006D:/ # find /data/adb/modules/testmodule/ /data/adb/modules/testmodule/ /data/adb/modules/testmodule/system /data/adb/modules/testmodule/system/product /data/adb/modules/testmodule/system/product/lib /data/adb/modules/testmodule/system/product/lib/.replace /data/adb/modules/testmodule/system/product/lib/libframesequence.so /data/adb/modules/testmodule/system/product/lib/libgiftranscode.so /data/adb/modules/testmodule/system/product/lib/newfile001 /data/adb/modules/testmodule/system/vendor /data/adb/modules/testmodule/system/vendor/app /data/adb/modules/testmodule/system/vendor/app/TimeService /data/adb/modules/testmodule/system/vendor/app/TimeService/TimeService.apk /data/adb/modules/testmodule/system/vendor/app/TimeService/oat /data/adb/modules/testmodule/system/vendor/app/TimeService/oat/arm64 /data/adb/modules/testmodule/system/vendor/app/TimeService/oat/arm64/TimeService.vdex /data/adb/modules/testmodule/system/vendor/app/TimeService/oat/arm64/TimeService.odex /data/adb/modules/testmodule/system/vendor/app/TimeService/newfile002 /data/adb/modules/testmodule/system/vendor/app/TimeService/.replace /data/adb/modules/testmodule/system/system_ext /data/adb/modules/testmodule/system/system_ext/app /data/adb/modules/testmodule/system/system_ext/app/FM2 /data/adb/modules/testmodule/system/system_ext/app/FM2/.replace /data/adb/modules/testmodule/system/system_ext/app/FM2/FM2.apk /data/adb/modules/testmodule/system/system_ext/app/FM2/lib /data/adb/modules/testmodule/system/system_ext/app/FM2/lib/arm64 /data/adb/modules/testmodule/system/system_ext/app/FM2/lib/arm64/libqcomfm_jni.so /data/adb/modules/testmodule/system/system_ext/app/FM2/oat /data/adb/modules/testmodule/system/system_ext/app/FM2/oat/arm64 /data/adb/modules/testmodule/system/system_ext/app/FM2/oat/arm64/FM2.odex /data/adb/modules/testmodule/system/system_ext/app/FM2/oat/arm64/FM2.vdex /data/adb/modules/testmodule/system/system_ext/app/FM2/newfile003 root@ASUS_I006D:/ # root@ASUS_I006D:/ # find /data/adb/modules/testmodule/system | grep newfile /data/adb/modules/testmodule/system/product/lib/newfile001 /data/adb/modules/testmodule/system/vendor/app/TimeService/newfile002 /data/adb/modules/testmodule/system/system_ext/app/FM2/newfile003 root@ASUS_I006D:/ # root@ASUS_I006D:/ # pwd / root@ASUS_I006D:/ # ls -l $( find /data/adb/modules/testmodule/system | grep newfile | cut -f6- -d "/" ) -rw-r--r-- 1 root root 0 2022-08-07 14:34 system/product/lib/newfile001 -rw-r--r-- 1 root root 0 2022-08-07 14:24 system/system_ext/app/FM2/newfile003 -rw-r--r-- 1 root root 0 2022-08-07 14:36 system/vendor/app/TimeService/newfile002 root@ASUS_I006D:/ #

See How to change any file or directory using Magisk for another approach to change files on read-only mounted filesystems.

Executing scripts while booting the phone

Magisk also supports executing additional scripts while booting the phone.

The scripts to be executed must be copied to one of these directories

/data/adb/post-fs-data.d/

/data/adb/service.d/

see the description in the original Magisk documentation: https://topjohnwu.github.io/Magisk/details.html

Working examples for this technique are described in these posts:

How to run a script at every boot using Magisk

How to disable or change the swap device in the Android 12 from ASUS for the Zenfone 8

How to create or change a swap device in the OmniROM 12 using Magisk

Another example for using this technique I found here https://gist.github.com/niikoo/3f6bd13a69f2d68f3dd51cc667e79bdc

# Boot logging # Create the file: /data/adb/post-fs-data.d/0001logcatboot #!/system/bin/sh mkdir -p /cache/logs /system/bin/logcat -r 1024 -n 9 -v threadtime -f /cache/logs/log >info.log 2>err.log &

Note that I added commands to create / cleanup the log directory and changed the logcat command on my phone:

#!/system/bin/sh mkdir -p /cache/logs rm -rf /cache/logs/* /system/bin/logcat -r 102400 -n 9 -v threadTime -f /cache/logs/log >/cache/logs/info.log 2>/cache/logs/err.log &

To execute additional scripts earlier in the boot process you must add them to the boot image -- see the section Using Magisk to unpack and repack the boot image below for details.

Creating Magisk Modules

The official documentation for creating a Magisk Module is here: https://topjohnwu.github.io/Magisk/guides.html

Creating Magisk Modules that only use the standard features isn't that difficult and the official documentation should be sufficient.

I suggest to download an existing simple Magisk Module; unzip the ZIP file and study the contents.

Using Magisk to unpack and repack the boot image

The binary magiskboot that is part of the Magisk package can be used to unpack and repack the boot partition for every phone supported by Magisk

For details see here: How to change files in the boot image using Magisk

Magisk also supports changing files in the root directory via Root Directory Overlays (see https://github.com/topjohnwu/Magisk/blob/master/docs/guides.md):
This feature can be used to create additional start or stop services for the Android OS:

Android uses init *rc files to define the services to start when booting and also to define the services to run when doing the shutdown (like the systemd or initd in other Linux implementations).
These files are read early in the boot process and therefore reside only in the ramdisk on the boot partition. To add new init* rc files the boot image must be modified. This can be done with Magisk:

Just add the new init*rc files and optimal other scripts or files to the boot image using Magisk . Magisk will then take care of processing the new init*rc files by the Android operating system when booting the phone.

See these posts:

How to run a script at shutdown

How to trigger an action when a property is changed .

How to enable root access using Magisk in a script

for examples for using that feature.

There are also magiskboot binaries for x86 in the Magisk apk file that can be used on a PC running the Linux OS:

see How to process Android boot image files on a PC running the Linux OS for more details about this feature.

Using Magisk to change the active slot

The Magisk App can also be used to change the active slot on phones with A/B slots -- for details see here :

How to switch the active slot

Directories and files used by Magisk

The directories and files used by Magisk are documented here https://topjohnwu.github.io/Magisk/guides.html

Some important directories and files for developing and trouble shooting Magisk Modules are:

The logfile used by Magisk is

/cache/magisk.log

In case something goes wrong with a Magisk Module you should first check that file.

The base directory for the data files for Magisk is

/data/adb

The config setttings for Magisk are stored in the SQLite database

/data/adb/magisk.db

To view (or probably) change the database entries via script you can either use sqlite3 binary (if installed).

Example:

ASUS_I006D:/ # sqlite3 /data/adb/magisk.db .tables denylist policies settings strings ASUS_I006D:/ # ASUS_I006D:/ # sqlite3 /data/adb/magisk.db SQLite version 3.7.6.3-Titanium Enter ".help" for instructions Enter SQL statements terminated with a ";" sqlite> .headers on sqlite> .mode column sqlite> sqlite> .tables denylist policies settings strings sqlite> sqlite> .schema CREATE TABLE denylist (package_name TEXT, process TEXT, PRIMARY KEY(package_name, process)); CREATE TABLE policies (uid INT, policy INT, until INT, logging INT, notification INT, PRIMARY KEY(uid)); CREATE TABLE settings (key TEXT, value INT, PRIMARY KEY(key)); CREATE TABLE strings (key TEXT, value TEXT, PRIMARY KEY(key)); sqlite> sqlite> select * from denylist ; sqlite> sqlite> select * from policies ; uid policy until logging notification ---------- ---------- ---------- ---------- ------------ 2000 2 0 1 1 10134 1 0 1 1 sqlite> sqlite> select * from settings ; key value ---------- ---------- denylist 0 su_biometr 0 sqlite> sqlite> select * from strings ; sqlite>

or use the magisk binary, e.g:
ASUS_I006D:/ # magisk --sqlite 'select * from policies' logging=1|notification=1|policy=2|uid=2000|until=0 ASUS_I006D:/ # ASUS_I006D:/ # magisk --sqlite 'select * from settings' key=denylist|value=0 key=su_biometric|value=0
ASUS_I006D:/ #

There is a handy little script to list all apps with root access in the XDA thread https://xdaforums.com/t/magisk-general-support-discussion.3432382/page-2681 :

list_apps_with_root_access.sh

#!/system/bin/sh

          

          GRANTED=$(magisk --sqlite "SELECT uid FROM policies WHERE
          policy = '2';")

          

          echo 'Packages with root granted:'

          

          for UID in $GRANTED

          do

          UID=$(echo $UID | sed 's!^uid=!!g')

          pm list packages --uid $UID

          done

How to create the Magisk database manually

To create the Magisk database manually (e.g. after installing Magisk manually via script) these commands can be used:

# start the Magisk daemon if necessary # MAGISK_DAEMON_IS_RUNNING=true /data/adb/magisk/magisk64 -v || /data/adb/magisk/magisk64 --daemon && MAGISK_DAEMON_IS_RUNNING=false # # force Magisk to create the Magisk database if it not yet exists # /data/adb/magisk/magisk64 --sqlite "PRAGMA user_version" # stop the Magisk daemon again # [ ${MAGISK_DAEMON_IS_RUNNING} = false ] && /data/adb/magisk/magisk64 --stop

To create the Magisk database using sqlite3 use these commands:

.open ${CUR_MAGISK_DATABASE_FILE} CREATE TABLE IF NOT EXISTS policies (uid INT, policy INT, until INT, logging INT,notification INT, PRIMARY KEY(uid)); CREATE TABLE IF NOT EXISTS settings (key TEXT, value INT, PRIMARY KEY(key)); CREATE TABLE IF NOT EXISTS strings (key TEXT, value TEXT, PRIMARY KEY(key)); CREATE TABLE IF NOT EXISTS denylist (package_name TEXT, process TEXT, PRIMARY KEY(package_name, process)); PRAGMA user_version=6; .exit

Note:

The Magisk database will be upgraded to the latest version automatically by the Magisk daemon after the next restart

Magisk root access configuration details

Magisk uses the table policies in the sqlite database /data/adb/magisk.db to store the list of root enabled apps.
See this post for details about this feature and how to enable root access via Magisk using a script.

Configuring the Magisk DenyList

see How to enable and configure the Magisk DenyList via cli commands for how to configure the Magisk DenyList via CLI commands

Magisk App installation directory

Use

pm list packages -f | grep magisk

to get the directory where the Magisk App is installed.

e.g.

ASUS_I006D:/dev/hP3B # pm list packages -f | grep magisk package:/data/app/~~oYhS9Dm94EFcjhYnyguTHg==/com.topjohnwu.magisk-2Ckcx1nejY3zYAklqVhBhw==/base.apk=com.topjohnwu.magisk ASUS_I006D:/dev/hP3B #

Credits: https://xdaforums.com/t/magisk-general-support-discussion.3432382/page-2689#post-87694851

The Magisk App Configuration files

The Magisk App stores the configuration (e.g. the app settings) in the default directory for App Settings, on my phone this is

/data/user_de/0/com.topjohnwu.magisk/shared_prefs

The file in that directory used to store the settings is

com.topjohnwu.magisk_preferences.xml

For more details about the file see the post Where does the Magisk App store the settings?.

Temporary Magisk config directory

While running Magisk uses a temporary directory for the configuration.

The path to the config directory can be retrieved via magisk binary:

ASUS_I006D:/dev/hP3B # magisk --path /dev/hP3B ASUS_I006D:/dev/hP3B#

The contents of that directory look like :

ASUS_I006D:/dev/hP3B #ls -al /dev/hP3B/ total 568 drwx------ 3 root root    200 1970-01-01 04:08 . drwxr-xr-x 27 root root   5480 2022-11-06 18:32 .. drwxr-xr-x 8 root root    180 1970-01-01 04:08 .magisk lrwxrwxrwx 1 root root     10 1970-01-01 04:08 magisk -> ./magisk64 -rwxr-xr-x 1 root root      0 1970-01-01 04:08 magisk32 -rwxr-xr-x 1 root root 247168 1970-01-01 04:08 magisk64 -rwxr-xr-x 1 root root 328240 1970-01-01 04:08 magiskpolicy lrwxrwxrwx 1 root root      8 1970-01-01 04:08 resetprop -> ./magisk lrwxrwxrwx 1 root root      8 1970-01-01 04:08 su -> ./magisk lrwxrwxrwx 1 root root     14 1970-01-01 04:08 supolicy -> ./magiskpolicy ASUS_I006D:/dev/hP3B # ls -al /dev/hP3B/.magisk/ total 7 drwxr-xr-x 8 root root 180 1970-01-01 04:08 . drwx------ 3 root root 200 1970-01-01 04:08 .. d--------- 2 root root 160 1970-01-01 04:08 block drwxr-xr-x 2 root root 7240 1970-01-01 04:08 busybox ---------- 1 root root 127 1970-01-01 04:08 config d--------- 7 root root 220 1970-01-01 04:08 mirror drwxr-xr-x 3 root root 3452 2022-11-06 18:32 modules drwxr-xr-x 2 root root    0 1970-01-01 04:08 pts d--------- 2 root root   80 1970-01-01 04:08 selinux ASUS_I006D:/dev/hP3B # cat /dev/hP3B/.magisk/config KEEPVERITY=false KEEPFORCEENCRYPT=false PATCHVBMETAFLAG=false RECOVERYMODE=false SHA1=1a05ccb9844d3ad4f6d1873dfbf76ebf83a5bdeb ASUS_I006D:/dev/hP3B #

Backups of boot partitions

Magisk creates backups of the boot partitions in sub directories in /data/adb with the name in the format

magisk_backup_[uniq_string]

e.g.:

Magisk creates backups of the boot partitions in /data/adb e.g.:

root@ASUS_I006D:/data/adb # ls -l /data/magisk_backup_* /data/magisk_backup_79f3370cd83d03441325998a8875888780c3182f: total 31712 -rwxr-xr-x 1 root root 32436260 2022-09-26 12:20 boot.img.gz /data/magisk_backup_a0c712541fd002c331c25772a3b8609ae2fba546: total 31712 -rwxr-xr-x 1 root root 32436965 2022-09-27 19:30 boot.img.gz root@ASUS_I006D:/data/adb #

uniq_string in the name of the backup files is the SHA-1 from the boot image that was patched to install Magisk:

e.g.

[xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ ./install_magisk_via_twrp.sh install_magisk_via_twrp.sh version - v2.0.0.1 - add Magisk to the boot partition of a phone running Android using TWRP .... Creating the boot image file "/sdcard/Download/boot_b.1086412.img" from the partition "/dev/block/by-name/boot_b" ... 196608+0 records in 196608+0 records out 100663296 bytes (96 M) copied, 0.313082 s, 307 M/s Checking the result ... -rw-rw---- 1 root media_rw 100663296 2022-11-06 16:01 /sdcard/Download/boot_b.1086412.img ... OK, patching the boot partition "/dev/block/by-name/boot_b" was successfull .... [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$

The image file patched by Magisk in this example is /sdcard/Download/boot_b.1086412.img.

The backup of the boot partition on the phone created by Magisk for this installation is:
ASUS_I006D:/ # ls -ld /data/magisk_backup* drwxr-xr-x 2 root root 3452 2022-11-06 17:14 /data/magisk_backup_1a05ccb9844d3ad4f6d1873dfbf76ebf83a5bdeb ASUS_I006D:/ # ASUS_I006D:/ # ls -l /data/magisk_backup_1a05ccb9844d3ad4f6d1873dfbf76ebf83a5bdeb total 50692 -rw-r--r-- 1 root root 51852324 2022-11-06 17:14 boot.img.gz ASUS_I006D:/ #

1a05ccb9844d3ad4f6d1873dfbf76ebf83a5bdeb is the SHA-1 from the image file used for the installation of Magisk:

ASUS_I006D:/ # sha1sum /sdcard/Download/boot_b.1086412.img 1a05ccb9844d3ad4f6d1873dfbf76ebf83a5bdeb /sdcard/Download/boot_b.1086412.img ASUS_I006D:/ # # or ASUS_I006D:/data/adb/workdir/unpack/ramdisk # /data/adb/magisk/magiskboot sha1 /sdcard/Download/boot_b.1086412.img 1a05ccb9844d3ad4f6d1873dfbf76ebf83a5bdeb ASUS_I006D:/data/adb/workdir/unpack/ramdisk #

The SHA1 from the previous boot image is stored in the file /dev/hP3B/.magisk/config used by Magisk while running, e.g.:

ASUS_I006D:/data/adb/workdir/unpack/ramdisk # cat /dev/hP3B/.magisk/config KEEPVERITY=false KEEPFORCEENCRYPT=false PATCHVBMETAFLAG=false RECOVERYMODE=false SHA1=1a05ccb9844d3ad4f6d1873dfbf76ebf83a5bdeb ASUS_I006D:/data/adb/workdir/unpack/ramdisk

/dev/hP3B is a directory on the temporary ramdisk used by Magisk while it is running.

Use the command magisk --path to retrieve the path for the current ramdisk while Magisk is running, e.g.:

ASUS_I006D:/data/adb/workdir/unpack/ramdisk # magisk --path /dev/hP3B ASUS_I006D:/data/adb/workdir/unpack/ramdisk #

The SHA1 from the previous boot image is also stored in the file .backup/.magisk in the ramdisk of a boot image, e.g:
# Note: unpack the boot image and the ramdisk from the boot image to get that file # ASUS_I006D:/data/adb/workdir/unpack/ramdisk #cat .backup/.magisk KEEPVERITY=false KEEPFORCEENCRYPT=false PATCHVBMETAFLAG=false RECOVERYMODE=false SHA1=1a05ccb9844d3ad4f6d1873dfbf76ebf83a5bdeb ASUS_I006D:/data/adb/workdir/unpack/ramdisk #

To do it all in once use:
grep "^SHA1=" $( magisk --path )/.magisk/config |cut -f2 -d "="

e.g.:

ASUS_I006D:/data/adb/workdir/unpack/ramdisk # grep "^SHA1=" $( magisk --path )/.magisk/config | cut -f2 -d "=" 1a05ccb9844d3ad4f6d1873dfbf76ebf83a5bdeb ASUS_I006D:/data/adb/workdir/unpack/ramdisk # #

Backup the Magisk config

To backup the Magisk config just copy the directories in /data/adb except the directory /data/adb/magisk (that directory is used for the binaries only), e.g.:

adb shell tar --exclude data/adb/magisk/ -czf /sdcard/Download/magisk_config_$( date +%Y-%m-%d).$$.tar /data/adb/
To restore the backup unpack the tar file on the phone and reboot the phone.

To also create a backup of the settings from the Magisk App create a backup of the directory with the Magisk App Settings (see above for details).

Start/Stop the Magisk App

To start the Magisk App via CLI command use

am start -n com.topjohnwu.magisk/.ui.MainActivity

To stop the Magisk App via CLI command use:

am force-stop com.topjohnwu.magisk

Start/Stop the Magisk Daemon

To stop the Magisk Daemon use

magisk --stop

Be aware that stopping the magisk daemon will remove all bind mounts for files in /system. To re-enable these bind mounts a reboot is required.

To start the Magisk Daemon use

/data/adb/magisk/magisk64 --daemon

To check if the Magisk Daemon is running use

/data/adb/magisk/magisk64 -v

Example output:

# Magisk Daemon is running ASUS_I006D:/ # /data/adb/magisk/magisk64 -v 25.2:MAGISK:R ASUS_I006D:/ # # Magisk Daemon is not running |ASUS_I006D:/ # /data/adb/magisk/magisk64 -v No daemon is currently running! 1|ASUS_I006D:/ #

Building Magisk

The Magisk source code is available at

https://github.com/topjohnwu/Magisk

There are also instructions how to create a local copy of the repository and compile Magisk on that page. I've successfully build Magisk using these instructions.

Miscellaneous

magiskboot can also be used compress or decompress files:

1|ASUS_I006D:/data/adb/magisk # ./magiskboot MagiskBoot - Boot Image Modification Tool Usage: ./magiskboot <action> [args...] Supported actions: ... compress[=format] <infile> [outfile]     Compress <infile> with [format] to [outfile].     <infile>/[outfile] can be '-' to be STDIN/STDOUT.     If [format] is not specified, then gzip will be used.     If [outfile] is not specified, then <infile> will be replaced     with another file suffixed with a matching file extension.     Supported formats: gzip zopfli xz lzma bzip2 lz4 lz4_legacy lz4_lg decompress <infile> [outfile]     Detect format and decompress <infile> to [outfile].     <infile>/[outfile] can be '-' to be STDIN/STDOUT.     If [outfile] is not specified, then <infile> will be replaced     with another file removing its archive format file extension.     Supported formats: gzip zopfli xz lzma bzip2 lz4 lz4_legacy lz4_lg 1|ASUS_I006D:/data/adb/magisk #

magiskboot is also used in TWRP to unpack and repack the boot image for installing TWRP into the boot partition -- see How to add additional files to an TWRP image.

Using the magisk binary while the phone is booted into TWRP

If the used TWRP can mount the volume for /data you can also use the binary magisk while in TWRP. The magisk binary is not in the path while booted into TWRP - therefor you must use the fully qualified filename:

This is

/data/adb/magisk/magisk64

for 64 Bit CPUs and

/data/adb/magisk/magisk32

for 32 Bit CPUs.

Some functions of Magisk are only usable if the Magisk daemon is running. To start the Magisk daemon the Magisk binary can also be used - example:

# read the policies table from the Magisk squlite database # ASUS_I006D:/ # /data/adb/magisk/magisk64 --sqlite "select * from policies ;" No daemon is currently running! # # -> the Magisk daemon is not running -> start it # 1|ASUS_I006D:/ # /data/adb/magisk/magisk64 --daemon ASUS_I006D:/ # ASUS_I006D:/ # /data/adb/magisk/magisk64 --sqlite "select * from policies ;" logging=1|notification=1|policy=2|uid=2000|until=0 logging=1|notification=1|policy=2|uid=10135|until=0 logging=1|notification=1|policy=2|uid=10143|until=0 logging=1|notification=1|policy=2|uid=10055|until=0 logging=1|notification=1|policy=2|uid=10142|until=0 ASUS_I006D:/ #

Use

/data/adb/magisk/magisk64 -V

to check if the Magisk daemon is running

Use

/data/adb/magisk/magisk64 --stop

to stop the Magisk Daemon, e.g.:

ASUS_I006D:/ # /data/adb/magisk/magisk64 -V 25200 # -> The Magisk Daemon is running ASUS_I006D:/ # /data/adb/magisk/magisk64 --stop ASUS_I006D:/ # ASUS_I006D:/ # /data/adb/magisk/magisk64 -V No daemon is currently running! 1|ASUS_I006D:/ # 2D

Trouble Shooting

An error like this

08-06 18:41:39.341 +0000 1356 1726 W ziparchive: Unable to open '/system/app/AsusFMRadio/AsusFMRadio.apk': Permission denied 08-06 18:41:39.341 +0000 1356 1726 E system_server: Failed to open APK '/system/app/AsusFMRadio/AsusFMRadio.apk': I/O error 08-06 18:41:39.354 +0000 1356 1356 W PackageManager: Failed to parse /system/app/AsusFMRadio: Failed to parse /system/app/AsusFMRadio/AsusFMRadio.apk

is most of the time caused by missing read permissions for the file.

Use

chmod o+r /system/app/AsusFMRadio/AsusFMRadio.apk

to fix it.

If something went wrong and booting the phone does not work anymore after installing a MagiskModule just remove the files in
/data/adb/modules/<modulename> and reboot the phone :

Either connect via adb to the not booting phone (this should be possible in most cases even if the boot process does not finish), delete the files, and reboot the phone. Or reboot the phone from a Recovery image like TWRP, delete the files in /data/adb/modules/<modulename>, and reboot the phone.

The same procedure can be used if booting the phone does not work anymore after adding another init script - just delete the new script in

/data/adb/post-fs-data.d

or

/data/adb/service.d

and reboot the phone

To catch errors from a script executed by Magisk you might use this technique
# redirect STDERR of all commands in the script to a file # exec 2>/data/script_stderr.log set -x ... rest of your script

To remove all installed Magisk Modules using the official method use:

magisk --remove-modules

to remove all modules (but not the new init scripts!) and reboot the phone

According to the FAQ Magisk will not start if the phone is booted into safe mode (see https://topjohnwu.github.io/Magisk/faq.html)

Be aware that after rebooting the phone again in normal mode all Magisk Modules are disabled and must be enabled again either using the Magisk App or via CLI command:

To reenable all Magisk Modules via shell command do

adb shell rm /data/adb/modules/*/disable

History of this entry

07.08.2022 /bs

Added additional infos about the permissions for new files for /system.
Added additional commands to the script for catching the OS logs while booting the phone
Added infos about a workaround to add new files to /product, /vendor, or /system_ext
Added infos about how to access file replaced by a Magisk Module

20.09.2022 /bs

Added new links to posts about configuring swap devices via Magisk script
Added a link to the post about how to use Magisk to unpack and repack the boot image

28.09.2022 /bs

Added infos about the backups of the boot partitions created by Magisk

30.09.2022 /bs

Added a short info about using Magisk Overlays to change files in the root filesystem Added the URL for the post with how to change the active slot using the Magisk App

02.10.2022 /bs

Added a short info about Root Directory Overlay system from Magisk

04.10.2022 /bs

Add an URL to another post to use the Root Directory Overlay system from Magsik
Fixed some spelling errors and also some errors in the code examples Added more details about changing files in /system

26.10.2022 /bs

Added the section Miscellaneous

28.10.2022/bs

Added the section Start/Stop the Magisk App Added the section Start/Stop the Magisk Daemon

02.11.2022 /bs

added the infos about the x86 version of the magiskboot executables in the Magisk apk file

fixed some spelling and formatting errors

04.11.2022 /bs

corrected some formatting errors
add the link to the Howto about making a file in /system writable

06.11.2022/bs

added more details about the boot partition backups created by Magisk

07.11.2022/bs

added more details about the boot partition backups created by Magisk
added the section about the temporary Magisk config directory

08.11.2022/bs

added the infos about how to get the Magisk app installation directory

25.11.2022/bs

added infos about how to build Magisk using a local copy of the repository

02.12.2022 /bs

added the infos about using the magisk binary while the phone is booted into TWRP

06.12.2022 /bs

added the section about Magisk root access configuration details
added infos about the files used to store the settings for the Magisk App

30.12.2022 /bs

added more infos about how to add new start / stop services in Android

07.05.2023 /bs

add infos about how to create the Magisk database manually

02.01.2024 /bs

added a link to the HowTo about how to configure the Magisk DenyList via CLI commands

How to change files in the boot image using Magisk

URL: https://xdaforums.com/t/how-to-change-files-in-the-boot-image-using-magisk.4495645/

How to changes files in the boot image using Magisk

Update 23.06.2024

See the section How to change files in the boot image using Magisk if there is no ramdisk in the boot partition for infos about boot partitions without a ramdisk.

See How to add files to the ramdisk used for the recovery boot for instructions to add additional files to the ramdisk used for recovery boots.

Sometime it's necessary to change a file in the boot image used for booting an Android phone (for example to change the default properties that are used before Magisk is loaded)

To change or add files to the boot image for an Android phone the Magisk binary magiskboot can be used

magiskboot is

A tool to unpack / repack boot images, parse / patch / extract cpio, patch dtb, hex patch binaries, and compress / decompress files with multiple algorithms.

and can handle the boot images for all phones supported by Magisk.

Warning

Be aware that changes to the boot image only survive until the installation of the next OS update. So it's strongly recommended to use Magisk to change files from the OS if possible.

Anyway -- to change a file in the boot image open a shell as user root on the phone and do:

Notes:

The following instructions were done on Zenfone 8 running OmniROM 12 but should also work for other ROMs.

At the end of this post are the commands to unpack the boot partition and to recreate the boot partition listed for cut & paste.

# # first create the directory tree for the work # # Note: # # This directory tree must be on a Unix filesystem like that mounted to /data; a filesystem like that mounted to /sdcard does not work # ASUS_I006D:/data/adb # cd /data/adb ASUS_I006D:/data/adb # mkdir workdir ASUS_I006D:/data/adb # mkdir workdir/backup ASUS_I006D:/data/adb # mkdir workdir/ramdisk ASUS_I006D:/data/adb # mkdir workdir/boot_files ASUS_I006D:/data/adb #
Next get the current boot slot:

ASUS_I006D:/data/adb/workdir # getprop ro.boot.slot_suffix _b

and create an image of the current boot partition:

# ASUS_I006D:/ # cd /data/adb/workdir ASUS_I006D:/data/adb/workdir # ASUS_I006D:/data/adb/workdir # dd if=/dev/block/by-name/boot_b of=./boot_b.img 196608+0 records in 196608+0 records out 100663296 bytes (96 M) copied, 0.374815 s, 256 M/s ASUS_I006D:/data/adb/workdir #

Create a backup of the original boot image file:

ASUS_I006D:/data/adb/workdir # cp boot_b.img backup/ ASUS_I006D:/data/adb/workdir #

Note:

You will need the boot image file just created to restore the boot partition in case the phone will not boot anymore with your changed boot partition. Therefor you should also copy the boot image file to your PC.

Next unpack the boot image using magiskboot into an empty directory:

ASUS_I006D:/data/adb/workdir # cd boot_files ASUS_I006D:/data/adb/workdir/boot_files # ls ASUS_I006D:/data/adb/workdir/boot_files # ASUS_I006D:/data/adb/workdir/boot_files # /data/adb/magisk/magiskboot unpack ../boot_b.img Parsing boot image: [../boot_b.img] HEADER_VER [3] KERNEL_SZ [42025472] RAMDISK_SZ [15093897] OS_VERSION [12.0.0] OS_PATCH_LEVEL [2022-05] PAGESIZE [4096] CMDLINE [] KERNEL_FMT [raw] RAMDISK_FMT [gzip] VBMETA ASUS_I006D:/data/adb/workdir/boot_files #

magiskboot will split the boot.img file into the various parts -- for most phones with Android 12 this should be the kernel and the ramdisk:

ASUS_I006D:/data/adb/workdir/boot_files # ls -l total 71264 -rw-r--r-- 1 root root 42025472 2022-09-18 18:22 kernel -rw-r--r-- 1 root root 30864336 2022-09-18 18:22 ramdisk.cpio ASUS_I006D:/data/adb/workdir/boot_files #

The files for the ramdisk used while booting the phone are in an cpio archive so to change them we must unpack the cpio archive with the ramdisk contents:

# ASUS_I006D:/ # cd /data/adb/workdir/ramdisk ASUS_I006D:/data/adb/workdir/ramdisk # 130|ASUS_I006D:/data/adb/workdir/ramdisk # cpio -idm <../boot_files/ramdisk.cpio ASUS_I006D:/data/adb/workdir/ramdisk # ls -ltr total 1868 -rw-r--r-- 1 root root 20700 1970-01-01 01:00 vendor_property_contexts -rw-r--r-- 1 root root 89884 1970-01-01 01:00 vendor_file_contexts drwxr-xr-x 2 root root 3452 1970-01-01 01:00 vendor drwxr-xr-x 2 root root 3452 1970-01-01 01:00 tmp -rw-r--r-- 1 root root 5783 1970-01-01 01:00 system_ext_property_contexts -rw-r--r-- 1 root root 3347 1970-01-01 01:00 system_ext_file_contexts drwxr-xr-x 2 root root 3452 1970-01-01 01:00 system_ext drwxr-xr-x 2 root root 3452 1970-01-01 01:00 sys drwxr-x--x 2 root root 3452 1970-01-01 01:00 storage -rw-r--r-- 1 root root 1027673 1970-01-01 01:00 sepolicy drwxr-xr-x 2 root root 3452 1970-01-01 01:00 second_stage_resources drwxr-xr-x 2 root root 3452 1970-01-01 01:00 sdcard -rw-r--r-- 1 root root 31390 1970-01-01 01:00 prop.default -rw-r--r-- 1 root root 1721 1970-01-01 01:00 product_property_contexts -rw-r--r-- 1 root root 306 1970-01-01 01:00 product_file_contexts drwxr-xr-x 2 root root 3452 1970-01-01 01:00 product drwxr-xr-x 2 root root 3452 1970-01-01 01:00 proc drwxr-xr-x 2 root root 3452 1970-01-01 01:00 postinstall -rw-r--r-- 1 root root 76958 1970-01-01 01:00 plat_property_contexts -rw-r--r-- 1 root root 40490 1970-01-01 01:00 plat_file_contexts drwxr-xr-x 2 root root 3452 1970-01-01 01:00 oem -rw-r--r-- 1 root root 0 1970-01-01 01:00 odm_property_contexts -rw-r--r-- 1 root root 0 1970-01-01 01:00 odm_file_contexts drwxr-xr-x 2 root root 3452 1970-01-01 01:00 mnt drwxr-xr-x 2 root root 3452 1970-01-01 01:00 metadata drwxr-xr-x 2 root root 3452 1970-01-01 01:00 linkerconfig -rwxr-x--- 1 root root 2025 1970-01-01 01:00 init.recovery.qcom.rc -rwxr-x--- 1 root root 491528 1970-01-01 01:00 init lrwxrwxrwx 1 root root 11 1970-01-01 01:00 etc -> /system/etc drwxr-xr-x 2 root root 3452 1970-01-01 01:00 dev lrwxrwxrwx 1 root root 12 1970-01-01 01:00 default.prop -> prop.default drwxr-xr-x 2 root root 3452 1970-01-01 01:00 debug_ramdisk drwxr-xr-x 2 root root 3452 1970-01-01 01:00 data_mirror drwxr-x--x 2 root root 3452 1970-01-01 01:00 data lrwxrwxrwx 1 root root 17 1970-01-01 01:00 d -> /sys/kernel/debug dr-xr-xr-x 2 root root 3452 1970-01-01 01:00 config lrwxrwxrwx 1 root root 50 1970-01-01 01:00 bugreports -> /data/user_de/0/com.android.shell/files/bugreports lrwxrwxrwx 1 root root 11 1970-01-01 01:00 bin -> /system/bin drwxr-xr-x 2 root root 3452 1970-01-01 01:00 batinfo drwxr-xr-x 2 root root 3452 1970-01-01 01:00 asdf drwxr-xr-x 2 root root 3452 1970-01-01 01:00 apex drwxr-xr-x 2 root root 3452 1970-01-01 01:00 acct drwxr-xr-x 2 root root 3452 1970-01-01 01:00 APD drwxr-xr-x 2 root root 3452 1970-01-01 01:00 ADF drwxr-xr-x 4 root root 3452 2022-09-18 18:28 first_stage_ramdisk drwxr-x--- 3 root root 3452 2022-09-18 18:28 overlay.d drwxr-xr-x 2 root root 3452 2022-09-18 18:28 odm_dlkm drwxr-xr-x 2 root root 3452 2022-09-18 18:28 odm drwxr-xr-x 3 root root 3452 2022-09-18 18:28 res drwxr-xr-x 5 root root 3452 2022-09-18 18:28 system drwxr-xr-x 2 root root 3452 2022-09-18 18:28 vendor_dlkm ASUS_I006D:/data/adb/workdir/ramdisk #

Now you can change the files from the ramdisk; add new files, or do anything necessary.

Be aware that most of the directories in the ramdisk are mount points to mount the other partitions. Therefor it's quite pointless to add files in these directories in the ramdisk.

Changing files in the ramdisk from the boot image

Changing one or more properties

Warning:

Please note that I am not an expert on Android internals and the following may be completely wrong.

(Any comments or corrections are welcome)

There are some files with property definitions in the boot image:

ASUS_I006D:/data/adb/workdir/ramdisk # find . -name "*.prop" ./default.prop ./system/etc/ramdisk/build.prop ASUS_I006D:/data/adb/workdir/ramdisk #

To change or add a property it should be sufficient to add or change it in the file default.prop in the ramdisk from the boot image. But that does not work (at least in my tests...) So to change or add a property it must be added to the file

/system/etc/ramdisk/build.prop

in the ramdisk for the boot image.

For example to add a swap device with 2 GB in the OmniROM these properties can be used:

# # define the size for the device zram0 (use the postfix G for values in Gigabytes or M for values in Megabytes) # vendor.zram.disksize=2147483648 vendor.zram.enable=1

The result after repacking and installing the boot image :

ASUS_I006D:/ # id uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 ASUS_I006D:/ # ASUS_I006D:/data/adb/workdir/ramdisk # getprop vendor.zram.disksize 2147483648 ASUS_I006D:/data/adb/workdir/ramdisk # vendor.zram.enable 1 ASUS_I006D:/data/adb/workdir/ramdisk # 1|ASUS_I006D:/data/adb/workdir/ramdisk # free total used free shared buffers Mem: 7612493824 4858556416 2753937408 12169216 15933440 -/+ buffers/cache: 4842622976 2769870848 Swap: 2147479552 0 2147479552 ASUS_I006D:/data/adb/workdir/ramdisk #

Notes:

These new properties defined in the file /system/etc/ramdisk/build.prop in the ramdisk are only visible for the root user. Most probably a SELinux definition in one of the *context* files in the boot image is required to make the property visible for all users (but I don't know the necessary permissions for the properties to make them visible for all user)

The current SELinx context for these properties is defined in the file vendor_property_contexts in the ramdisk from boot image:

ASUS_I006D:/data/adb/workdir/ramdisk # grep zram vendor_property_contexts vendor.zram.disksize u:object_r:vendor_exported_system_prop:s0 vendor.zram.enable u:object_r:vendor_exported_system_prop:s0 persist.vendor.zram u:object_r:vendor_mpctl_prop:s0 vendor.asus.zram u:object_r:vendor_mpctl_prop:s0 vendor.zram. u:object_r:vendor_mpctl_prop:s0 ASUS_I006D:/data/adb/workdir/ramdisk #

To add a new property without a specific SELInux context it seems to be sufficient to add the property to the file /system/etc/ramdisk/build.prop in the ramdisk from the boot image
(this does not work for properties starting with vendor. ):

# add the property to the file # 130|ASUS_I006D:/ # tail -3 /data/adb/workdir/ramdisk/system/etc/ramdisk/build.prop build.mysettings.variable01.value=5555 # # end of file ASUS_I006D:/ #

After the reboot the property is visible for non-root user:

ASUS_I006D:/ $ id uid=2000(shell) gid=2000(shell) groups=2000(shell),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),1078(ext_data_rw),1079(ext_obb_rw),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats),3009(readproc),3011(uhid),3012(readtracefs) context=u:r:shell:s0 ASUS_I006D:/ $ ASUS_I006D:/ $ getprop build.mysettings.variable01.value    5555 ASUS_I006D:/ $

But the new property can only be changed by the user root:

ASUS_I006D:/ $ iduid=2000(shell) gid=2000(shell)groups=2000(shell),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),1078(ext_data_rw),1079(ext_obb_rw),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats),3009(readproc),3011(uhid),3012(readtracefs) context=u:r:shell:s0 ASUS_I006D:/ $ ASUS_I006D:/ $ setprop build.mysettings.variable01.value 7777Failed to set property 'build.mysettings.variable01.value' to '7777'. See dmesg for error reason. 1|ASUS_I006D:/ $ 1|ASUS_I006D:/ $ su - ASUS_I006D:/ # ASUS_I006D:/ # id uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 ASUS_I006D:/ # ASUS_I006D:/ # setprop build.mysettings.variable01.value 7777    ASUS_I006D:/ # ASUS_I006D:/ # getprop build.mysettings.variable01.value 7777 ASUS_I006D:/ # ASUS_I006D:/ # ^D ASUS_I006D:/ $ id uid=2000(shell) gid=2000(shell) groups=2000(shell),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),1078(ext_data_rw),1079(ext_obb_rw),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats),3009(readproc),3011(uhid),3012(readtracefs) context=u:r:shell:s0ASUS_I006D:/ $ getprop build.mysettings.variable01.value 7777 ASUS_I006D:/ $

(see here https://source.android.com/docs/core/architecture/configuration/add-system-properties for details about defining properties)

When done with your changes repack the ramdisk.

First we recreate the cpio archive:

ASUS_I006D:/data/adb/workdir # cd /data/adb/workdir/ramdisk ASUS_I006D:/data/adb/workdir/ramdisk # ASUS_I006D:/data/adb/workdir/ramdisk # find . | cpio -o >../boot_files/ramdisk.cpio ASUS_I006D:/data/adb/workdir/ramdisk # ASUS_I006D:/data/adb/workdir/ramdisk # ls -l ../boot_files/ramdisk.cpio -rw-r--r-- 1 root root 30865560 2022-09-18 18:31 ../boot_files/ramdisk.cpio ASUS_I006D:/data/adb/workdir/ramdisk #

And then we recreate the boot image with the kernel and the changed ramdisk using magiskboot:

ASUS_I006D:/data/adb/workdir/ramdisk # cd /data/adb/workdir/boot_files/ ASUS_I006D:/data/adb/workdir/boot_files # 130|ASUS_I006D:/data/adb/workdir/boot_files # /data/adb/magisk/magiskboot repack ../boot_b.img ../boot_b.img.new Parsing boot image: [../boot_b.img] HEADER_VER [3] KERNEL_SZ [42025472] RAMDISK_SZ [15093897] OS_VERSION [12.0.0] OS_PATCH_LEVEL [2022-05] PAGESIZE [4096] CMDLINE [] KERNEL_FMT [raw] RAMDISK_FMT [gzip] VBMETA Repack to boot image: [../boot_b.img.new] HEADER_VER [3] KERNEL_SZ [42025472] RAMDISK_SZ [15094405] OS_VERSION [12.0.0] OS_PATCH_LEVEL [2022-05] PAGESIZE [4096] CMDLINE [] ASUS_I006D:/data/adb/workdir/boot_files # ASUS_I006D:/data/adb/workdir/boot_files # ls -l ../boot_b.img ../boot_b.img.new -rw-r--r-- 1 root root 100663296 2022-09-18 18:20 ../boot_b.img -rw-r--r-- 1 root root 100663296 2022-09-18 18:34 ../boot_b.img.new ASUS_I006D:/data/adb/workdir/boot_files #

The new boot image file can now be written to the boot partition (either via dd or fastboot):

ASUS_I006D:/data/adb/workdir # cd /data/adb/workdir ASUS_I006D:/data/adb/workdir # dd if=./boot_b.img.new of=/dev/block/by-name/boot_b 196608+0 records in 196608+0 records out 100663296 bytes (96 M) copied, 5.324559 s, 18 M/s ASUS_I006D:/data/adb/workdir #

That's it -- now the new boot image should be used for the next reboots of the phone.

Additional infos

Please note that the file /system/etc/ramdisk/build.prop does not exist in the ramdisk from the boot disk used in the Original Android 12 for the ASUS Zenfone 8 . So this approach to change the properties does not work in that OS.
But the general approach used in these instructions to change files in the ramdisk used for the boot image does also work in the Original Android 12 for the ASUS Zenfone 8.

In the OmniROM for the Zenfone 8 and also in the Original Android 10 for the Zenfone 8 the swap device is created and configured with this init script: /vendor/bin/init.asus.zram.sh

See here

How to disable or change the swap device in the Android 12 from ASUS for the Zenfone 8

for a method using Magisk to change the swap device config in the Original ASUS Android 12

See here

How to disable or change the swap device in the Android 12 from ASUS for the Zenfone 8

for a method using Magisk to change the swap device config in the OnmiROM 12

Replace files in the root directory

Magisk also supports changing files in the root directory via an overlay system -- see the section Root Directory Overlay System in this page:

https://github.com/topjohnwu/Magisk/blob/master/docs/guides.md

To use this feature it's also neccessary to unpack and repack the boot partition.

An example about how to use this feature is here: Magisk overlay - execute a script or copy files

Another example is in this thread: https://github.com/topjohnwu/Magisk/issues/2978
Including some sample code to use magiskboot to change files in the ramdisk image from the boot partition:

./magiskboot unpack boot_root.img ./magiskboot cpio ramdisk.cpio \ "mkdir 0700 overlay.d" \ "add 0700 overlay.d/init.custom.rc init.custom.rc" \ "mkdir 0700 overlay.d/sbin" \ "add 0700 overlay.d/sbin/custom.sh init.custom.sh" ./magiskboot repack boot_root.img

Another example for this techniquie is in this post:

How to enable access via adb on a new installed OS

Trouble Shooting

If the phone does not boot anymore with the changed boot partition restore the boot partition from the boot image file created at the beginning either via dd after booting from a recovery image like TWRP or using fastboot to flash the boot paritition.

It's important to at least unpack the cpio with the ramdisk contents on an Unix filesystem (e.g. ext3, ext4, etc) - this can not be done in the device mounted to /sdcard.

Instructions for cut & paste

The instructions below are for cut & paste. Be aware that there is no error checking.

Instructions to copy the boot partition to a boot image file and to unpack the boot image and the ramdisk

CUR_SLOT=$( getprop ro.boot.slot_suffix ) cd /data/adb mkdir /data/adb/workdir mkdir /data/adb/workdir/backup mkdir /data/adb/workdir/ramdisk mkdir /data/adb/workdir/boot_files cd /data/adb/workdir dd if=/dev/block/by-name/boot${CUR_SLOT} of=./boot${CUR_SLOT}.img cp boot_b.img backup/ cd boot_files /data/adb/magisk/magiskboot unpack ../boot${CUR_SLOT}.img cd /data/adb/workdir/ramdisk cpio -idm <../boot_files/ramdisk.cpio pwd ls -ltr

Instructions to repack the ramdisk and the boot image and to rewrite the boot partition

CUR_SLOT=$( getprop ro.boot.slot_suffix ) cd /data/adb/workdir/ramdisk/ find . | cpio -o >../boot_files/ramdisk.cpio cd /data/adb/workdir/boot_files/ /data/adb/magisk/magiskboot repack ../boot${CUR_SLOT}.img ../boot${CUR_SLOT}.img.new cd /data/adb/workdir dd if=./boot${CUR_SLOT}.img.new of=/dev/block/by-name/boot${CUR_SLOT} reboot

History of this entry

History

30.09.2022/bs

add infos about changing files in the root directory via Magisk overlays

02.11.2022/bs

see How to process Android boot image files on a PC running the Linux OS for instructions about using magiskboot on a machine running the Linux OS

30.12.2022 /bs

Added a link to the new post to enable adb access

How to disable or change the swap device in the Android 12 from ASUS for the Zenfone 8

URL: https://xdaforums.com/t/how-to-disable-or-change-the-swap-device-in-the-android-12-from-asus-for-the-zenfone-8.4495085/

How to disable or change the swap device in the Android 12 from ASUS for the Zenfone 8

In the original Android 12 from ASUS for the Zenfone 8 a swap device is configured by default:

ASUS_I006D:/ # getprop ro.product.build.fingerprint asus/WW_I006D/ASUS_I006D:12/SKQ1.210821.001/31.1010.0411.113:user/release-keys ASUS_I006D:/ # ASUS_I006D:/ # getprop ro.product.build.date Wed Jul 27 02:56:23 CST 2022 ASUS_I006D:/ # ASUS_I006D:/ # free total used free shared buffers Mem: 7612669952 7389499392 223170560 84582400 12218368 -/+ buffers/cache: 7377281024 235388928 Swap: 4294963200 0 4294963200 ASUS_I006D:/ #

Note:

All commands in these instructions must be done by the user root.

To disable the swap device in the Android 12 from ASUS open a shell and execute:

setprop vendor.zram.enable 0

Example:

ASUS_I006D:/data/adb/workdir/ramdisk # free total used free shared buffers Mem: 7612669952 7334137856 278532096 82571264 19410944 -/+ buffers/cache: 7314726912 297943040 Swap: 4294963200 0 4294963200 root@ASUS_I006D:/data/adb/workdir/ramdisk # ASUS_I006D:/ # setprop vendor.zram.enable 0 ASUS_I006D:/ # ASUS_I006D:/ # free total used free shared buffers Mem: 7612669952 7392669696 220000256 84586496 12263424 -/+ buffers/cache: 7380406272 232263680 Swap: 0 0 0 ASUS_I006D:/ #

Note:

To execute the command via adb use

adb shell su - -c setprop vendor.zram.enable 0

To re-enable the swap device execute as user root :

setprop vendor.zram.enable 1

e.g.

ASUS_I006D:/ # free total used free shared buffers Mem: 7612669952 7392796672 219873280 84598784 12288000 -/+ buffers/cache: 7380508672 232161280 Swap: 0 0 0 ASUS_I006D:/ # ASUS_I006D:/ # setprop vendor.zram.enable 1 ASUS_I006D:/ # ASUS_I006D:/ # free total used free shared buffers Mem: 7612669952 7397068800 215601152 84598784 12292096 -/+ buffers/cache: 7384776704 227893248 Swap: 4294963200 0 4294963200 ASUS_I006D:/ #

To change the size of the swap device change the value for the variable

vendor.zram.disksize

and disable and re-enable the swap device. The default unit for the value of the variable vendor.zram.disksize is byte; use the postfix M for megabytes or G for gigabytes.

e.g. to use a swap device with 2 GB do:

ASUS_I006D:/ # free total used free shared buffers Mem: 7612669952 7398703104 213966848 84631552 12365824 -/+ buffers/cache: 7386337280 226332672 Swap: 4294963200 0 4294963200 ASUS_I006D:/ # ASUS_I006D:/ # setprop vendor.zram.enable 0 ASUS_I006D:/ # ASUS_I006D:/ # free total used free shared buffers Mem: 7612669952 7394570240 218099712 84631552 12361728 -/+ buffers/cache: 7382208512 230461440 Swap: 0 0 0 ASUS_I006D:/ # ASUS_I006D:/ # setprop vendor.zram.disksize 2G ASUS_I006D:/ # ASUS_I006D:/ # free total used free shared buffers Mem: 7612669952 7533645824 79024128 86020096 12369920 -/+ buffers/cache: 7521275904 91394048 Swap: 0 0 0 ASUS_I006D:/ # ASUS_I006D:/ # setprop vendor.zram.enable 1 ASUS_I006D:/ # ASUS_I006D:/ # free total used free shared buffers Mem: 7612669952 7514472448 98197504 85999616 12374016 -/+ buffers/cache: 7502098432 110571520 Swap: 2147479552 0 2147479552 ASUS_I006D:/ #

Note:

This can also be done in a Magisk script to make the new config for the swap device persistent:

e.g.

cat <<EOT >/data/adb/service.d/0010reconfigure_swap # # change the size of the swap volume to 2 GB # setprop vendor.zram.enable 0 setprop vendor.zram.disksize 2G setprop vendor.zram.enable 1 EOT chmod 755 /data/adb/service.d/0010reconfigure_swap

Result after a reboot:

ASUS_I006D:/ $ free total used free shared buffers Mem: 7612669952 6047076352 1565593600 58441728 17780736 -/+ buffers/cache: 6029295616 1583374336 Swap: 2147479552 0 2147479552 ASUS_I006D:/ $

To disable the swap device completely via Magisk script use these commands:

cat <<EOT >/data/adb/service.d/0010disable_swap # # disable the swap device # setprop vendor.zram.enable 0 EOT chmod 755 /data/adb/service.d/0010disable_swap

Result after a reboot:

shell@ASUS_I006D:/ $ free total used free shared buffers Mem: 7612669952 5930426368 1682243584 55574528 17719296 -/+ buffers/cache: 5912707072 1699962880 Swap: 0 0 0shell@ASUS_I006D:/ $

Note that this approach does not work in the OmniROM 12 for the ASUS Zenfone 8.

History of this entry

Update 20.09.2022/bs

see How to create or change a swap device in the OmniROM using Magisk for how to change or disable the swap device in the OmniROM.

Update 04.10.2022 /bs

See How to trigger an action when a property is changed for how to implement enabling or disabling the swap device on ZRAM via a property value in the OmniROM or other OS

How to create a swap device in the OmniROM 12 using Magisk

URL: https://xdaforums.com/t/how-to-create-or-change-a-swap-device-in-the-omnirom-using-magisk.4495415/

How to create a swap device in the OmniROM 12 using Magisk

The init script in the OmniROM running on the ASUS Zenfone 8 to create a swap device is:

/vendor/bin/init.asus.zram.sh

This script will create a swap device while booting the OS if that functionality is enabled.

To check if there is a swap device configured use the OS command free:

root@ASUS_I006D:/ # free total used free shared buffers Mem: 7612493824 4732076032 2880417792 12857344 15298560 -/+ buffers/cache: 4716777472 2895716352 Swap: 4294963200 0 4294963200 root@ASUS_I006D:/ #

If the result of the script is not okay you can create a Magisk script to correct the config for the swap device.

The script /vendor/bin/init.asus.zram.sh uses the property

vendor.zram.enable

to determine if a swap device should be created and the property

vendor.zram.disksize

for the (new) size for the swap device.

If the property vendor.zram.enable is set to 1 but the property vendor.zram.disksize is not set the script uses a default value for the size of the swap device depending on the amount of installed memory.

To create a swap device via a Magisk Script (or change the config of the swap device created by default) it's sufficient to just set the two properties to the approbiate values and execute the init script , e.g.

Note:

This must be done as user root.

cat >/data/adb/service.d/0010create_swap_device.sh <<EOT #!/system/bin/sh # there is a missing leading slash (/) in one of the commands in the file /vendor/bin/init.asus.zram.sh # therefor the working directory when executing the script must be / # cd / # # enable creating the swap device # setprop vendor.zram.enable 1 # # use a 512 MB swap device # The default unit for vendor.zram.disksize is bytes; use the prefix M for megabytes or G for gigabytes # # Do not set this property to use the default value for the swap device (which is depending on the installed memory) # setprop vendor.zram.disksize 512M # # now (re)create the swap device # /vendor/bin/init.asus.zram.sh EOT chmod 755 /data/adb/service.d/0010create_swap_device.sh

To disable the swap device at all use this Magisk script:

cat >/data/adb/service.d/0010disable_swap_device.sh <<EOT #!/system/bin/sh # there is a missing leading slash (/) in one of the commands in the file /vendor/bin/init.asus.zram.sh # therefor the working directory when executing the script must be / # cd / # # enable creating the swap device (-> necessary to delete an existing swap device) # setprop vendor.zram.enable 1 # new size for the swap device (0 = do not configure a swap device) # setprop vendor.zram.disksize 0 # # now (re)create the swap device (-> only delete an existing swap device) # /vendor/bin/init.asus.zram.sh EOT chmod 755 /data/adb/service.d/0010disable_swap_device.sh

Note:

See How to disable or change the swap device in the Android 12 from ASUS for the Zenfone 8 for how to reconfigure swap in the Original Android 12 from ASUS for the Zenfone 8.

Please note that the device /dev/block/zram0 is created in the RAM but the memory for the device /dev/block/zram0 device is only allocated on demand - e.g.:

# # free memory with a free swap device on zram0 with 6 GB is about 2783735808 (about 2 GB) # root@ASUS_I006D:/ # free total used free shared buffers Mem: 7612489728 4828753920 2783735808 13639680 15556608 -/+ buffers/cache: 4813197312 2799292416 Swap: 6442446848 0 6442446848 root@ASUS_I006D:/ # root@ASUS_I006D:/ # setprop vendor.zram.disksize 0 root@ASUS_I006D:/ # /vendor/bin/init.asus.zram.sh root@ASUS_I006D:/ # # # free memory with no swap device on zram0 is about 2802741248 (about 2 GB) # root@ASUS_I006D:/ # free total used free shared buffers Mem: 7612489728 4809748480 2802741248 13750272 15577088 -/+ buffers/cache: 4794171392 2818318336 Swap: 0 0 0 root@ASUS_I006D:/ # root@ASUS_I006D:/ # setprop vendor.zram.disksize 4G root@ASUS_I006D:/ # root@ASUS_I006D:/ # /vendor/bin/init.asus.zram.sh Swapspace size: 4194300k, UUID=4a914375-e28f-4181-850a-87cc792eacbe root@ASUS_I006D:/ # # # free memory with a free swap device on zram0 with 4 GB is about 2773835776 (about 2 GB) # root@ASUS_I006D:/ # free total used free shared buffers Mem: 7612489728 4838653952 2773835776 13877248 15581184 -/+ buffers/cache: 4823072768 2789416960 Swap: 4294963200 0 4294963200 root@ASUS_I006D:/ #

History of this entry

Update 04.10.2022 /bs

See How to trigger an action when a property is changed for how to implement enabling or disabling the swap device on ZRAM via a property value in the OmniROM or other OS

How to switch the active slot

URL: https://xdaforums.com/t/how-to-manually-switch-the-active-slot.4499789/

How to switch the active slot

Switching the active slot on phones with A/B slots is normally only used by the updater to install a new OS update (see https://source.android.com/docs/core/ota/ab for details)

This functionality can also be quite useful if you have two different operating systems on your phone and want to switch between them without reinstalling.

Since the data on all partitions that do not belong to the A/B partitions is used by both operating systems (the one in slot A and the one in slot B), this functionality is limited by the installed operating systems.

The data in most partitions that are only present once should be usable for all operating systems (see here), but this does not apply to the data in the partition that is mounted to /data.

Switching the active slot does work if the OS (including the OS version) installed in slot A and B are equal. It does work most of the time if the OS installed in slot A and B are equal but with different OS versions.

And it should work if the OS installed in slot A and slot B use different directories in /data, e.g. in slot A is a Android based OS installed and in slot B a not on Android based OS (like for example
pocketMarketOS)

It does not work (most of the times) if there are different Android distributions in slot A and B: e.g. I did not get it to work if the OS in slot A is the original Android 12 from ASUS and the OS in slot B is OmniROM 12. (Probably it would work after removing one or more OS related directories from /data but I do not know which - infos on how to use these OS in parallel are welcome)

The instructions below were done on an ASUS Zenfone 8, but I believe they should work with other phones with A/B slots also.

Please note that you should have a working TWRP image for your phone on hand to restore the active slot in case of boot problems.

To get the current slot while the OS is running use the shell command getprop, e.g. execute in a shell on the phone or in an adb shell:

root@ASUS_I006D:/ # getprop ro.boot.slot_suffix _a root@ASUS_I006D:/ #

To get the active slot in fastboot mode or while the phone is booted into the bootloader execute on the PC connected to the phone:

fastboot getvar current-slot
e.g.

[ OmniRomDev - xtrnaw7@t15g /data/develop/android ] $ sudo fastboot getvar current-slot current-slot: b Finished. Total time: 0.001s [ OmniRomDev - xtrnaw7@t15g /data/develop/android ] $
There are various methods to manual switch the active slot:

1. Switching the slot using fastboot

Switching the slot via fastboot works if TWRP is installed in the boot partition.

To use it connect the phone to an USB port of the PC and issue these commands on the PC:

# boot the phone into fastboot mode either if in the bootloader # sudo fastboot reboot fastboot # # or if the phone is booteind into the recovery or in the normal mode # adb reboot fastboot # wait until the phone is booted in fastboot mode # get the current slot # sudo fastboot getvar current-slot # change the current slot # sudo fastboot set_active b # check the result # sudo fastboot getvar current-slot # reboot the phone from new current-slot # sudo fastboot reboot

Example

[xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ sudo fastboot reboot Rebooting OKAY [ 0.000s] Finished. Total time: 0.201s [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ adb reboot fastboot [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ sudo fastboot getvar current-slot current-slot: a Finished. Total time: 0.001s [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ sudo fastboot set_active b Setting current slot to 'b' OKAY [ 0.042s] Finished. Total time: 0.043s [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ sudo fastboot getvar current-slot current-slot: b Finished. Total time: 0.000s [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ sudo fastboot reboot Rebooting OKAY [ 0.000s] Finished. Total time: 0.251s # check the result # [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ adb shell getprop ro.boot.slot_suffix _b [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$

Notes:

Switching the active slot in fastboot mode does not work if the boot image from the original Android 12 from ASUS or from the OmniROM is installed:

[xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ sudo fastboot set_active b Setting current slot to 'b' FAILED (remote: 'Unable to set slot') fastboot: error: Command failed [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$

The same is true it the phone is booted into the bootloader:

[ OmniRomDev - xtrnaw7@t15g /data/develop/android/test ] $ sudo fastboot --set-active=a Setting current slot to 'a' FAILED (remote: 'Slot Change is not allowed in Lock State') fastboot: error: Command failed [ OmniRomDev - xtrnaw7@t15g /data/develop/android/test ] $

Changing the lock state via fastboot flashing lock|unlock triggers a factory reset.

2. Switching the slot using TWRP

The active slot for the next reboot can also be changed in the reboot dialog from TWRP.
This can also be done if the phone is booted from the TWRP image.

Note:

This is the fallback to get the phone booting again in case the reboot of the other slot fails for whatever reason.

3. Switching the Slot via Magisk

The active slot can also be changed in the Magisk App. This is used to update the boot partition after an OTA (see here https://topjohnwu.github.io/Magisk/ota.html for details) but can also be used to manual change the active slot.

Open the Magisk App and choose the "install" button to install Magisk into the boot partition. In the next dialog use the method "Install to inactive Slot (After OTA)".

This works also if no update (OTA) was installed and it does not change the Magisk config - it just replaces the Magisk files in the inactive boot partition and activates the inactive boot partition. To reboot the phone then from the now active partition press the "Reboot" button after the update of the boot partition by Magisk is done.

This only works if the OS in both slots is supported by Magisk, of course.

4. Using the binary bootctl

There is binary called bootctl that can be used to switch the active slot. The source code of bootctl is part of the source for the Android OS but as far as I know the binary is not compiled by default and therefor also not installed on the phone.

But there is Magisk module to install the bootctl binary; that Magisk Module can be downloaded from here:

https://github.com/roihershberg/bootctl-binary

Unfortunately the current version of the Magisk Module with bootctl does not install and work out of the box on Android 12 or OmniROM 12 for the ASUS Zenfone 8. Therefor I created a corrected version of the Magisk Module.

The corrected Magisk Module can be downloaded here:

https://bnsmb.de/files/public/Android/bootctl-binary-v2.1.1.zip

The Changelog for the corrected Magisk module for bootctl is:

bootctl changelog

Update 21.09.2022 /bs

- changed the max. API version in customize.sh to 32 so that the module can be installed in Android 12

- renamed the binary to bootctl.bin

- added the wrapper script bootctl to start bootctl because the directory /vendor/lib64 with the necessary libraries for the binary is not in the default library path in Android 12

bootctl must be executed by the root user in a shell on the phone or in an adb shell.

To change the active slot with bootctl use

bootctl set-active-boot-slot SLOT

SLOT is either 0 (for slot A) or 1 (for slot B); the next reboot will then boot from that slot.

To get the active booted slot with bootctl use

bootctl get-current-slot

Notes:

The usage help for bootctl is:

root@ASUS_I006D:# bootctl -h ./bootctl - command-line wrapper for the boot HAL. Usage: ./bootctl COMMAND Commands: hal-info - Show info about boot_control HAL used. get-number-slots - Prints number of slots. get-current-slot - Prints currently running SLOT. mark-boot-successful - Mark current slot as GOOD. get-active-boot-slot - Prints the SLOT to load on next boot. set-active-boot-slot SLOT - On next boot, load and execute SLOT. set-slot-as-unbootable SLOT - Mark SLOT as invalid. is-slot-bootable SLOT - Returns 0 only if SLOT is bootable. is-slot-marked-successful SLOT - Returns 0 only if SLOT is marked GOOD. get-suffix SLOT - Prints suffix for SLOT. set-snapshot-merge-status STAT - Sets whether a snapshot-merge of any dynamic partition is in progress. Valid STAT values are: none, unknown, snapshotted, merging, or cancelled. get-snapshot-merge-status - Prints the current snapshot-merge status. SLOT parameter is the zero-based slot-number.

The source code for bootctl is here:

https://android.googlesource.com/platform/system/extras/+/master/bootctl/bootctl.cpp

https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/boot_control.h

There is also a simple app to change the active slot via the bootctl binary: Switch-My-Slot-Android

Note that the app can only be installed after the Magisk Module for bootctl is installed.

Trouble Shooting

If you get an error like this after rebooting from the previously inactive slot

Cannot load Android System. Your data may be corrupt. if you continue o get this message, you may need to perform a factory data reset and erase all user data storeed on this device
or if the phone boots only in the bootloader after the reboot from the previously inactive slot boot the phone from the TWRP image to activate the previous active slot again.

e.g. for the phone ASUS Zenfone 8:

sudo fastboot boot /data/backup/ASUS_ZENFONE8/twrp/twrp-3.6.1_12-1-I006D.img

Then in TWRP select to reboot the phone and in the dialog to reboot switch the slot back to the previous active slot.

You can ignore the error messages printed by TWRP after changing the active slot - just continue rebooting the phone.

Note:

On the ASUS Zenfone 8 Press the buttons Up, Down, and Power at the same time for about 10 or more seconds to turn off the phone. Afterwards you can boot into the bootloader.

History of this entry

06.11.2022 /bs

The section about changing the slot in fastboot mode was not correct: fastboot mode and bootloader mode are differents states of the phone.

06.11.2022 /bs (2)

Switching the slot in fastboot mode does only work if TWRP is installed in the boot partition

12.01.2024 /bs

In TWRP version 3.7.0_12-1 for the ASUS Zenfone 8 the bootctl binary is not included anymore (the bootctl binary is part of the image for TWRP 3.7.0_12-0 and TWRP 3.6.1_12-1).

How to run a script at shutdown

URL: https://xdaforums.com/t/how-to-run-a-script-at-shutdown.4500719/

How to run a script at shutdown

Update 20.06.2024

See the section How to change files in the boot image using Magisk if there is no ramdisk in the boot partition for infos about boot partitions without a ramdisk.

To define additional startup scripts via Magisk the Magisk directories /data/adb/service.d and /data/adb/post-fs-data.d can be used. Unfortunately there is no equivalent for scripts that should be executed during shutdown.

So we must use other methods to implement these kind of scripts.

Using the overlay feature of Magisk to run a script at shutdown

Introduction

in Android it is possible to define actions that will be executed when certain conditions are satisfied. These definitions are done in the file init.rc (and other .rc files) using the Android Init Language.

And this feature can be used to execute a command when the phone is shutting down.

Note:

For details about the Android Init Language used for these files see here https://android.googlesource.com/platform/system/core/+/master/init/README.md

The .rc files used by Android are in the directories

/system/etc/init
/vendor/etc/init
/odm/etc/init

Note: The first .rc file read is /system/etc/init/hw/init.rc

Unfortunately it's useless to change the .rc files in these directories using the Magisk features to change files in the directory /system because these files are processed by the OS before the new files are "created" by Magisk.

Therefor the overlay functionality from Magisk must be used to create additional .rc files (see the section Root Directory Overlay System on this page https://github.com/topjohnwu/Magisk/blob/master/docs/guides.md for details about this Magisk Feature).

Preparation

To be able to restore the original boot partition in case of an error create an image of the original boot partition from the phone on your PC before starting the development:

CUR_SLOT=$( adb shell getprop ro.boot.slot_suffix ) adb shell su - -c dd if=/dev/block/by-name/boot${CUR_SLOT} | cat >boot${CUR_SLOT}.img

e.g.

[ OmniRomDev - xtrnaw7@t15g /data/develop/android/test ] $ CUR_SLOT=$( adb shell getprop ro.boot.slot_suffix ) [ OmniRomDev - xtrnaw7@t15g /data/develop/android/test ] $ echo ${CUR_SLOT} _b [ OmniRomDev - xtrnaw7@t15g /data/develop/android/test ] $ [ OmniRomDev - xtrnaw7@t15g /data/develop/android/test ] $ adb shell su - -c dd if=/dev/block/by-name/boot${CUR_SLOT} | cat >boot${CUR_SLOT}.img 196608+0 records in 196608+0 records out 100663296 bytes (96 M) copied, 2.668147 s, 36 M/s [ OmniRomDev - xtrnaw7@t15g /data/develop/android/test ] [ OmniRomDev - xtrnaw7@t15g /data/develop/android/test ] $ ls -ltr boot${CUR_SLOT}.img -rw-r--r--. 1 xtrnaw7 xtrnaw7 100663296 Oct 1 12:13 boot_b.img [ OmniRomDev - xtrnaw7@t15g /data/develop/android/test ] $

To trouble shoot issues with this approach it is highly recommended to create an Magisk init script in the directory

/data/adb/post-fs-data.d

to fetch and store the Android logs into a persistent file. Use these commands to create the script:

cat >/data/adb/post-fs-data.d/0001logcatboot <<-EOT mkdir -p /cache/logs # backup the OS logs from before the reboot: # [ -r /cache/logs/log ] && mv /cache/logs/log /cache/logs/oldlog /system/bin/logcat -r 102400 -n 9 -v threadTime -f /cache/logs/log >/cache/logs/info.log 2>/cache/logs/err.log & EOT chmod 755 /data/adb/post-fs-data.d/0001logcatboot
Using this script the log messages from before the last reboot are stored in the file /cache/logs/oldlog.
To activate the script the phone must be rebooted.

Check the contents of the directory /cache/logs/log after the reboot as user root to be sure that it works.

shell@ASUS_I006D:/ $ su - -c ls -ltr /cache/logs total 205008 -rw-rw-rw- 1 root root       0 1970-01-06 08:16 info.log -rw-rw-rw- 1 root root       0 1970-01-06 08:16 err.log -rw-r----- 1 root root 4707523 2022-10-01 17:29 log shell@ASUS_I006D:/ $

Details

The trigger in the .rc files for the action that should be done while shutting down is

on shutdown

The trigger can be used more then once; the OS will execute all defined actions for the trigger in the order they are found in the rc files.

The action to run an executable in the .rc file is

exec [ <seclabel> [ <user> [ <group>\* ] ] ] -- <command> [ <argument>\* ]

Fork and execute command with the given arguments. The command starts after â€œ--â€ so that an optional security context, user, and supplementary groups can be provided. No other commands will be run until this one finishes. seclabel can be a - to denote default. Properties are expanded within argument. Init halts executing commands until the forked process exits.

In Android SELinux is enabled by default. Therefor it's neccessary to use the correct SELinux context for the files used.

(Note: The SELinux context for the init process executing the action is u:r:init:0.)

It's quite difficult to find the correct SELinux contexts in Android for this approach therefor it's better to use the general SELinux context defined by Magisk: u:r:magisk:s0 .

Implementation

Note:

All commands must be done as user root in an session on the phone or in an adb session.

So first create the neccessary directories and files:

mkdir -p /data/init_scripts mkdir -p /data/init_scripts/log

Create the script to execute on shutdown:

cat >/data/init_scripts/my_shutdown.sh <<-\EOT #!/system/bin/sh SHUTDOWN_LOG="/data/init_scripts/log/myshutdown.$$.log" echo "$0: Shutdown with parameter \"$*\" started at $( date ) " >>${SHUTDOWN_LOG} echo "*** id : " >>${SHUTDOWN_LOG} 2>&1 id >>${SHUTDOWN_LOG} 2>&1 # ... add necessary commands ... EOT chmod 755 /data/init_scripts/my_shutdown.sh

Correct the SELinux context:

chcon -R u:r:magisk:s0 /data/init_scripts/

Check the result

root@ASUS_I006D:/ # find /data/init_scripts/ -exec ls -lZd {} \; drwxr-xr-x 3 root root u:r:magisk:s0 3452 2022-10-01 16:12 /data/init_scripts/ -rwxr-xr-x 1 root root u:r:magisk:s0 637 2022-10-01 16:12 /data/init_scripts/my_shutdown.sh drwxr-xr-x 2 root root u:r:magisk:s0 3452 2022-10-01 16:16 /data/init_scripts/log root@ASUS_I006D:/ #

Create a working dir:

# # create a working directory # mkdir -p /data/adb/workdir

Now create the additional .rc file:

# # change the current directory to the working directory # cd /data/adb/workdir cat >init.custom.rc <<-\EOT on shutdown     exec u:r:magisk:s0 -- /system/bin/sh /data/init_scripts/my_shutdown.sh 0008 on early-init    setprop my_custom_rc_file loaded EOT

Note:

The additional trigger is for testing the new .rc file (see the trouble shooting section below for details)
Magisk supports more then one .rc file; the name of the .rc file is meaningless but the extension must be .rc

And now add the new file to the ramdisk on the boot partition:

# # change the current directory to the working directory # cd /data/adb/workdir # get the current active slot # CURRENT_SLOT=$( getprop ro.boot.slot_suffix ) echo "The current active slot is: ${CURRENT_SLOT}" # copy the boot partition from the active slot to a file # dd if=/dev/block/by-name/boot${CURRENT_SLOT} of=./boot_root.img # unpack the image file # /data/adb/magisk/magiskboot unpack ./boot_root.img # add the new dirs and files to the ramdisk from the boot partition # /data/adb/magisk/magiskboot cpio ramdisk.cpio \ "mkdir 0700 overlay.d" \ "add 0700 overlay.d/init.custom.rc init.custom.rc" # recreate the image file for the boot partition # /data/adb/magisk/magiskboot repack boot_root.img # write the corrected image file to the boot partition # dd if=./new-boot.img of=/dev/block/by-name/boot${CURRENT_SLOT}

Note:

The commands to unpack and pack the ramdisk manually using the cpio command are (if NOT using the Magisk binary magiskboot):

RAMDISK=$PWD/ramdisk mkdir ${RAMDISK} cd ${RAMDISK} # unpack the ramdisk # cpio -idm <../ramdisk.cpio # ... do what ever is necessary with the files/dirs in ${RAMDISK} # pack the ramdisk again # cd ${RAMDISK} find . | cpio -o >../ramdisk.cpio

Now reboot the phone to activate the new .rc config and after the reboot check that the .rc file was processed

getprop my_custom_rc_file

e.g

shell@ASUS_I006D:/ $ getprop my_custom_rc_file loaded shell@ASUS_I006D:/ $

If the property defined in the .rc file, my_custom_rc_file, is not set something went wrong and you should check the OS logs and double check your config.

If the new property is defined you can test the shutdown action by rebooting the phone again and check the output directory.

While doing this reboot the new shutdown script should be executed and after the reboot is done there should be the log files from the shutdown script:

shell@ASUS_I006D:/ $ su -        root@ASUS_I006D:/ # ls -l /data/init_scripts/log total 0 -rw------- 1 root root 179 2022-10-01 18:23 myshutdown.4617.log root@ASUS_I006D:/ # cat /data/init_scripts/log/myshutdown.4617.log                                                                                                                                                        /data/init_scripts/my_shutdown.sh: Shutdown with parameter "0008" started at Sat Oct 1 18:23:14 CEST 2022 *** id : uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 root@ASUS_I006D:/ #

That's it.

Note that you can change the script executed while doing the shutdown without changing the boot image again.
But you should always test the script before rebooting -- an error in your script may stop the reboot.

To change the additional .rc files it's necessary to recreate the ramdisk and boot partition.

The filesystems for /data and for /sdcard are still mounted while executing the actions for the trigger "on shutdown" .

To log the current environment while executing the shutdown script you can add code like this to the script:

( echo echo "*** Environment while executing the shutdown script ..." echo echo "*** pwd: " pwd echo echo "*** id: " id echo echo "*** df -h: " df -h echo echo "*** ps -efZ : " ps -efZ echo echo "*** env: " env echo echo "*** set: " set echo ) >>/data/init_scripts/log/myshutdown_env.log 2>&1

To create a directory in which other actions from the .rc file (like write) can write with enabeld SELinux use one of the SELInux contexts the init process can write to, e.g:

mkdir /data/system_data chcon u:object_r:system_data_file:s0 /data/system_data

Now the .rc config

on shutdown write /data/system_data/myshutdown.log Shutdown_started\n

will work.

See the file /plat_file_contexts in the ramdisk from the boot partition for other existing SELinux contexts:

root@ASUS_I006D:/data/adb/test # /data/adb/magisk/magiskboot cpio ramdisk.cpio "extract plat_file_contexts plat_file_contexts" Loading cpio: [ramdisk.cpio] Extract [plat_file_contexts] to [plat_file_contexts] root@ASUS_I006D:/data/adb/test # ls -l plat_file_contexts-rw-r--r-- 1 root root 40490 2022-10-03 16:27 plat_file_contexts root@ASUS_I006D:/data/adb/test #

Please be aware that these changes will be gone after the next OS update. But on the other hand it's quite easy to create a script to re install the shutdown script without user intervention.

For further hints regarding *rc files see also Hints for adding files to the boot partition

Trouble Shooting

The main reason for problems with this approach are invalid SELinux contexts. Therefor you should test your script in permissive SELinux mode if it does not work like expected. To do that temporary disable SELinux before rebooting (SELinux will be automatically enabled again after the reboot), e.g.:

# set SELinux to permissive # setenforce 0 reboot

and check the log messages in the directory /cache/logs/oldlog for SELinux related messages:

su - -c grep deny /cache/logs/oldlog

Note that you can not disable SELinux in an action in an .rc file.

To check if your additional .rc file is processed by Magisk add a statement like these to the custom .rc file in the overlay directory:

on early-init     setprop sys.example.foo bar

If this statement is processed by Magisk and Android the property sys.example.foo should be defined after the reboot, e.g.:

root@ASUS_I006D:/ # getprop sys.example.foo bar root@ASUS_I006D:/ #

To check if the "on shutdown" trigger is processed use :

on shutdown    write /sdcard/Download/myshutdown.log Shutdown_started\n

and reboot with disabled SELinux

setenforce 0 reboot

If the "on shutdown" trigger in your .rc file is processed there should exist the file

/sdcard/Download/myshutdown.log

after the reboot

If the shutdown of the phone hangs open another adb session to the phone and kill the script (the adb daemon should still run while the shutdown script is running)

If the phone does not boot anymore with the new shutdown script reboot the phone from the TWRP image and fix / delete the new shutdown script. Or reflash the boot partition with the image file created before starting the development.

In general you should carefully check your .rc file for syntax errors -- entries in the file after the first syntax error will be ignored

Useful URLs

I used ideas and code from the web pages listed below for this HowTo:

How to run an executable on boot and keep it running?

How to run an Android init service with superuser SELinux context?

Magisk overlay - execute a script or copy files

History of this entry

03.10.2022 /bs

added code about how to extract a single file (plat_file_contexts) from the ramdisk cpio image using magiskboot 23.06.2024 /bs

fixed some typos

How to trigger an action when a property is changed

URL: https://xdaforums.com/t/how-to-trigger-an-action-when-a-property-is-changed.4501489/

How to trigger an action when a property is changed

See the section How to change files in the boot image using Magisk if there is no ramdisk in the boot partition for infos about boot partitions without a ramdisk.

In Android it's possible to trigger various actions by changing the value of a property. This feature is quite handy and the implementation using Magisk is not really difficult.

As an example

In the original Android from ASUS for the Zenfone 8 you can disable and enable the swap on a ZRAM device by changing the value for the property vendor.zram.enable:

e.g.:

To turn the swap on use

setprop vendor.zram.enable 1
and to turn the swap off use

setprop vendor.zram.enable 0

To get the current value use

getprop vendor.zram.enable

or

check the output of the OS command free.

(see How to disable or change the swap device in the Android 12 from ASUS for the Zenfone 8 for details)

This feature is not implemented in the OmniROM for the ASUS Zenfone 8 but quite useful so let's see how to implement it in the OmniROM.

The Triggers and Action for this Android feature are configured in the init.rc files in the root filesystem for the OS (see https://android.googlesource.com/platform/system/core/+/master/init/README.md for details). The root filesystem for Android is read-only mounted so without creating your own Android image for the phone it's not possible to add the functionality to the OS.

But we can use the Root Directory Overlay System from Magisk (see https://github.com/topjohnwu/Magisk/blob/master/docs/guides.md for the documentation) to implement it.

The detailed process for creating additional *.rc files for Android via Magisk is described here:

How to run a script at shutdown

Therefore I will not go into the details here. But please read that post before you continue

First we check how this feature is implemented in the Original Android for the Zenfone:

Enabling and disabling the swap device on ZRAM is configured in the .rc file

/vendor/etc/hw/init.asus.debugtool.rc

using these settings in the original Android OS for the Zenfone 8:

service asus_zram /system/vendor/bin/sh /vendor/bin/init.asus.zram.sh     user root     group root     disabled     seclabel u:r:vendor_qti_init_shell:s0     oneshot on property:persist.vendor.zram.enable=1     setprop vendor.zram.enable "1"     setprop vendor.zram.disksize ${persist.vendor.zram.disksize} on property:persist.vendor.zram.enable=0     setprop vendor.zram.enable "0"     setprop vendor.zram.disksize ${persist.vendor.zram.disksize} on property:vendor.zram.enable=*     start asus_zram

The script used to enable or disable the swap device on ZRAM in the original Android for the Zenfone 8 is:

/vendor/bin/init.asus.zram.sh

The script is quite simple (see also below):

It uses two properties to configure the swap device:

vendor.zram.enable : if this property is set to 1 the script enables the swap device and if the property is set to 0 it disables the swap device

vendor.zram.disksize : the value of this property is the size of the ramdisk.

Now we can implement this feature for the OmniROM:

Note:

All commands must be done as user root in a shell on the phone or in an adb shell

First we check the prerequisites for the feature:

The used shell for the service exists in the OmniROM:

root@ASUS_I006D:/data/adb/workdir # ls -Zl /vendor/bin/sh -rwxr-xr-x 1 root shell u:object_r:vendor_shell_exec:s0 318216 2009-01-01 01:00 /vendor/bin/sh root@ASUS_I006D:/data/adb/workdir #

The script to toggle the ramdisk on the swap device also already exists in the OmniROM:

root@ASUS_I006D:/data/adb/test # ls -l /vendor/bin/init.asus.zram.sh -rwxr-xr-x 1 root shell 1127 2009-01-01 01:00 /vendor/bin/init.asus.zram.sh root@ASUS_I006D:/data/adb/test #

And the necessary SELinux contexts are also already defined in the OmniROM.

So, let's start:

# create a temporary directory # mkdir /data/adb/workdir cd /data/adb/workdir # create the additional .rc file # cat >init.asus.zram.rc <<-\EOT # # Note: # # The service definition for an OS without the script init.asus.zram.sh should be # # service asus_zram /system/bin/sh /system/sbin/init.asus.zram.sh # service asus_zram /system/vendor/bin/sh /vendor/bin/init.asus.zram.sh     user root     group root     disabled     seclabel u:r:vendor_qti_init_shell:s0     oneshot # # these properties can be used to define the initial state of the ramdisk on ZRAM # on property:persist.vendor.zram.enable=1     setprop vendor.zram.enable "1"     setprop vendor.zram.disksize ${persist.vendor.zram.disksize} on property:persist.vendor.zram.enable=0     setprop vendor.zram.enable "0"     setprop vendor.zram.disksize ${persist.vendor.zram.disksize} on property:vendor.zram.enable=*     start asus_zram # The property persist.vendor.zram.enable is not defined in the OmniROM. If necessary you can add another trigger in the .rc file, # e.g to enable the swap device by default use # on early-init    setprop persist.vendor.zram.enable 1 EOT # get the current active slot # CURRENT_SLOT=$( getprop ro.boot.slot_suffix ) echo "The current active slot is: ${CURRENT_SLOT}" # copy the boot partition from the active slot to a file dd if=/dev/block/by-name/boot${CURRENT_SLOT} of=./boot_root.img # unpack the image file /data/adb/magisk/magiskboot unpack ./boot_root.img # add the new dir and file to the ramdisk from the boot partition /data/adb/magisk/magiskboot cpio ramdisk.cpio \ "mkdir 0700 overlay.d" \ "add 0700 overlay.d/init.custom.rc init.asus.zram.rc" # recreate the image file for the boot partition /data/adb/magisk/magiskboot repack boot_root.img # write the corrected image file to the boot partition dd if=./new-boot.img of=/dev/block/by-name/boot${CURRENT_SLOT}

That's it.

After the next reboot switching the ramdisk on the ZRAM device via an property should be active:

root@ASUS_I006D:/ # reboot
# .....
# start a new adb session
# .... shell@ASUS_I006D:/ $ getprop ro.omni.version 12-20220703-zenfone8-MICROG # # Note: The property is only visible to the user root # shell@ASUS_I006D:/ $ su - root@ASUS_I006D:/ # id uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 root@ASUS_I006D:/ # root@ASUS_I006D:/ # free         total        used        free      shared     buffers Mem:       7612493824 4835229696 2777264128    14585856    14487552 -/+ buffers/cache:     4820742144 2791751680 Swap:      4294963200           0 4294963200 root@ASUS_I006D:/ # root@ASUS_I006D:/ # getprop vendor.zram.enable 1 root@ASUS_I006D:/ # # -> now disable the swap on ZRAM root@ASUS_I006D:/ # setprop vendor.zram.enable 0 root@ASUS_I006D:/ # root@ASUS_I006D:/ # getprop vendor.zram.enable 0 root@ASUS_I006D:/ # root@ASUS_I006D:/ # free         total        used        free      shared     buffers Mem:       7612493824 4840824832 2771668992    14524416    14483456 -/+ buffers/cache:     4826341376 2786152448 Swap:               0           0           0 root@ASUS_I006D:/ # # -> now enable the swap on ZRAM again root@ASUS_I006D:/ # setprop vendor.zram.enable 1 root@ASUS_I006D:/ # root@ASUS_I006D:/ # getprop vendor.zram.enable 1 root@ASUS_I006D:/ # root@ASUS_I006D:/ # free         total        used        free      shared     buffers Mem:       7612493824 4844777472 2767716352    14524416    14512128 -/+ buffers/cache:     4830265344 2782228480 Swap:      4294963200           0 4294963200 root@ASUS_I006D:/ #

See the post How to enable access via adb on a new installed OS for another usage example for this technique.

Workarounds for other configuration

Find below some workarounds for OS versions without the prerequisites for implementing this feature.

1. The script to toggle the swap /vendor/bin/init.asus.zram.sh does not exist

If the script /vendor/bin/init.asus.zram.sh does not exist in your OS create a Magisk Module for the script

To create a (dummy) Magisk Module for the script do:

mkdir -p /data/adb/modules/toggle_ram/system/bin # # create the script to toggle the ramdisk on ZRAM # cat >/data/adb/modules/toggle_ram/system/bin/init.asus.zram.sh <<-\EOT lahaina_set=`getprop vendor.asus.zram_setting` if test "$lahaina_set" != "1"; then     echo "[asus_zram] init.kernel.post_boot-lahaina.sh not finished yet!"> /dev/kmsg     exit 0 fi disksize=`getprop vendor.zram.disksize` zram_enable=`getprop vendor.zram.enable` MemTotalStr=`cat /proc/meminfo | grep MemTotal` MemTotal=${MemTotalStr:16:8} let RamSizeGB="( $MemTotal / 1048576 ) + 1" if test "$disksize" = ""; then     disksize="4096M" fi echo "[asus_zram]RamSizeGB=${RamSizeGB}" > /dev/kmsg if test "$zram_enable" = "1"; then     if [ $RamSizeGB -le 7 ]; then #this is for 6G; or the value will be 4G(8G,12G,16G,18G,etc)         disksize="( $RamSizeGB * 1024 ) / 2""M"     fi     swapoff /dev/block/zram0 2>/dev/kmsg     echo 1 > sys/block/zram0/reset 2>/dev/kmsg     sleep 1     echo lz4 > /sys/block/zram0/comp_algorithm     echo $disksize > /sys/block/zram0/disksize 2>/dev/kmsg     mkswap /dev/block/zram0 2>/dev/kmsg     swapon /dev/block/zram0 -p 32758 2>/dev/kmsg     echo "[asus_zram]write zram disksize=${disksize}" > /dev/kmsg fi if test "$zram_enable" = "0"; then     swapoff /dev/block/zram0 2>/dev/kmsg     echo "[asus_zram]turn off the zram" > /dev/kmsg fi EOT chmod 755 /data/adb/modules/toggle_ram/system/bin/init.asus.zram.sh chown root:shell /data/adb/modules/toggle_ram/system/bin/init.asus.zram.sh chcon u:object_r:vendor_file:s0 /data/adb/modules/toggle_ram/system/bin/init.asus.zram.sh

Check the result:

root@ASUS_I006D:/ # ls -lZ /data/adb/modules/toggle_ram/system/bin/init.asus.zram.sh-rwxr-xr-x 1 root shell u:object_r:vendor_file:s0 1109 2022-10-03 21:14 /data/adb/modules/toggle_ram/system/bin/init.asus.zram.sh root@ASUS_I006D:/ #

Then change the script to execute in the service definition in the .rc file to /system/bin/init.asus.zram.sh:

service asus_zram /system/vendor/bin/sh /system/bin/init.asus.zram.sh

The rest of the instructions can be be used without changes.

2. The binary /system/vendor/bin/sh does not exist

If the shell /system/vendor/bin/sh does not exist in the OS create an Magisk Module with an approbiate shell:

Note:

The shell /system/bin/sh can not be used because it's configured with another SELinux context:

root@ASUS_I006D:/ # ls -lZ /system/vendor/bin/sh -rwxr-xr-x 1 root shell u:object_r:vendor_shell_exec:s0 318216 2009-01-01 01:00 /system/vendor/bin/sh root@ASUS_I006D:/ # # but root@ASUS_I006D:/data/adb/workdir # ls -Zl /system/bin/sh -rwxr-xr-x 1 root shell u:object_r:shell_exec:s0 307768 2009-01-01 01:00 /system/bin/sh root@ASUS_I006D:/data/adb/workdir #

To create a (dummy) Magisk Module for the necessary shell do:

mkdir -p /data/adb/modules/vendorshell/system/bin cp /system/bin/sh /data/adb/modules/vendorshell/system/bin/sh chcon u:object_r:vendor_shell_exec:s0 /data/adb/modules/vendorshell/system/bin/vendor_sh

Check the result:

root@ASUS_I006D:/data/adb/workdir # ls -lZ /data/adb/modules/vendorshell/system/bin/vendor_sh -rwxr-xr-x 1 root root u:object_r:vendor_shell_exec:s0 307768 2022-10-03 20:41 /data/adb/modules/vendorshell/system/bin/vendor_sh root@ASUS_I006D:/data/adb/workdir #

Now change the shell in the service definition in the .rc file to /system/bin/vendor_sh:

service asus_zram /system/bin/vendor_sh /vendor/bin/init.asus.zram.sh

The rest of the instructions can be be used without changes.

3. The necessary shell /system/vendor/bin/sh and the script /vendor/bin/init.asus.zram.sh do both not exist

If the necessary shell /system/vendor/bin/sh and the script /vendor/bin/init.asus.zram.sh do not exist you can also create one (dummy) Magisk module for both.

See

https://topjohnwu.github.io/Magisk/guides.html

and

Some hints for using Magisk on Android phones

for instructions and infos about how to create a real Magisk Module.

4. Missing SELinux contexts

These instructions assume that all necessary SELinux contexts are already defined in the used OS.
If there are SELinux contexts missing it might not work without adding the missing SELinux contexts (that might be difficult and I must admit that I did not test that)

5. only for the records:

The magiskboot command can also be used to add new files to the /sbin directory in the root filesystem.

This can be can be configured like this:

/data/adb/magisk/magiskboot cpio ramdisk.cpio \ "mkdir 0700 overlay.d" \ "add 0700 overlay.d/init.custom.rc init.asus.zram.rc" \ "mkdir 0700 overlay.d/sbin" \ "add 0700 overlay.d/sbin/my_new_script.sh my_script.sh"

This is useful for adding binaries, scripts. or data files for actions that should be executed while the other partitions are not yet mounted and therefor the files in these filesystems not yet available. But for the action defined in this post this is not necessary.

And be aware that the files added to the root filesystem ( /sbin/my_new_script.sh in this example) are only available while booting the phone.

6. Other problems

In case of problems please check the post

How to run a script at shutdown

again. Also check the restrictions for this approach documented in that post.

How to install TWRP via script

URL: https://xdaforums.com/t/how-to-install-twrp-via-script.4511355/

How to install TWRP via script

The recovery image TWRP is very useful for the maintenance of the Android OS. You can either boot the phone from the TWRP image without installing it using these commands:

adb reboot bootloader sudo fastboot boot ./twrp-3.7.0_12-0-I006D.img
or install the TWRP image into the boot partition and use the command
adb reboot recovery

or

sudo fastboot boot recovery

to boot into TWRP. Depending on the OS used there is also an option in the reboot dialog in the GUI to reboot into the recovery.

While the phone is booted from TWRP you can open a shell as user root via adb.

The only official method to install TWRP into the boot partition of the phone is via the TWRP GUI, e.g.:

boot the phone from the TWRP image (see above)
install TWRP into the boot partition using the menu entry "Advanced" and then "Flash Current TWRP". TWRP will then install itself into both boot partitions.

There is no method to install TWRP without user interaction via a script (at least I did not found any ...). Therefor I wrote a shell script to install TWRP without user input:

install_twrp_from_within_twrp.sh

The script install_twrp_from_within_twrp.sh can be used to install TWRP either into a boot partition or into an image file for the boot partition. The script must be executed in a shell on the phone while the phone is booted from the TWRP image that should be installed.

The usage of the script is:

ASUS_I006D:/data/local/tmp $ ./install_twrp_from_within_twrp.sh -h install_twrp_from_within_twrp.sh - script to install TWRP into a boot partition or into a file Usage: install_twrp_from_within_twrp.sh [-f] [boot_image_file] [new_boot_image_file] boot_image can be either an image file from the boot partition or a boot partition block device (e.g. /dev/block/by-name/boot_a) new_boot_partition_file must be the absolute name of a file. Both parameter are optional If the script is called without a parameter TWRP will be installed in the current boot partition. If boot_image_file is a block device and new_boot_image_file is missing TWRP will be installed in the block device. If the parameter new_boot_image_file is used the script will only write a boot partition image with TWRP into that file. Use the parameter -f to ignore missing or invalid checksums. This script must run in a shell in TWRP Environment variables used by the script if set: BIN_DIR (Current value: "/system/bin") TMPDIR (Current value: "/data/local/tmp") |ASUS_I006D:/data/local/tmp $

The commands used in the script install_twrp_from_within_twrp.sh to install TWRP are copied from the TWRP source file with the code to install TWRP:

https://github.com/TeamWin/android_bootable_recovery/blob/android-12.1/twrpRepacker.cpp

Note that TWRP contains a copy of the binary magiskboot to unpack and repack the boot partition. Therefor the installation should work on all phones that are supported by Magisk.
Be aware that the installation of TWRP will wipe out the installation of Magisk (or any other tool for enabling root access). So for installing TWRP and Magisk they must be installed in this order:

install TWRP
install Magisk (or a similar tool)

Example usage:

Example for install_twrp_from_within_twrp.sh

ASUS_I006D:/ # /cache/install_twrp_from_within_twrp.sh /dev/block/by-name/boot_b Installing TWRP into a boot image or boot partition via script Checking the running OS ... OK, running a shell in TWRP: "twrp_I006D" version "3.7.0_12-0" Checking the prerequisites for installing TWRP ... Using the partition "/dev/block/by-name/boot_b" as source for the installation of TWRP Creating the new boot image with TWRP in the file "/data/local/tmp/img.568/new-boot.img" Checking if the required executables exist ... OK, the file "/system/bin/magiskboot" exists and is executable OK, the file "/system/bin/cpio" exists and is executable OK, the file "/system/bin/sha256sum" exists and is executable OK, the file "/system/bin/getprop" exists and is executable OK, the file "/system/bin/dd" exists and is executable Checking if the requried data files exist ... OK, the file "/ramdisk-files.sha256sum" exists OK, the file "/ramdisk-files.txt" exists OK, the file "/dev/block/by-name/boot_b" exists Checking the check sums of the files for the new ramdisk ... OK, the check sums of the files for the new ramdisk are okay Directory "/data/local/tmp" successfully created Directory "/data/local/tmp/img.568" successfully created Directory "/data/local/tmp/img_files.568" successfully created The temporary directory to unpack the boot image is "/data/local/tmp/img_files.568" Unpacking the boot image from "/dev/block/by-name/boot_b" ... Parsing boot image: [/dev/block/by-name/boot_b] HEADER_VER [3] KERNEL_SZ [42027520] RAMDISK_SZ [34080747] OS_VERSION [12.0.0] OS_PATCH_LEVEL [2022-05] PAGESIZE [4096] CMDLINE [] KERNEL_FMT [raw] RAMDISK_FMT [gzip] VBMETA OK, "/dev/block/by-name/boot_b" successfully unpacked to "/data/local/tmp/img_files.568" : Creating a backup of the original ramdisk ... total 74412 -rw-rw-rw- 1 root root 50 2022-10-24 19:35 header -rw-r--r-- 1 root root 42027520 2022-10-24 19:35 kernel -rw-r--r-- 1 root root 34080747 2022-10-24 19:35 ramdisk.cpio.org Creating the new ramdisk with TWRP ... New ramdisk with TWRP "/data/local/tmp/img_files.568/ramdisk.cpio" successfully created. -rw-rw-rw- 1 root root 85852380 2022-10-24 19:35 /data/local/tmp/img_files.568/ramdisk.cpio -rw-r--r-- 1 root root 34080747 2022-10-24 19:35 /data/local/tmp/img_files.568/ramdisk.cpio.org Repacking the boot image into the file "/data/local/tmp/img.568/new-boot.img" ... + cd /data/local/tmp/img_files.568 + /system/bin/magiskboot repack /dev/block/by-name/boot_b /data/local/tmp/img.568/new-boot.img Parsing boot image: [/dev/block/by-name/boot_b] HEADER_VER [3] KERNEL_SZ [42027520] RAMDISK_SZ [34080747] OS_VERSION [12.0.0] OS_PATCH_LEVEL [2022-05] PAGESIZE [4096] CMDLINE [] KERNEL_FMT [raw] RAMDISK_FMT [gzip] VBMETA Repack to boot image: [/data/local/tmp/img.568/new-boot.img] HEADER_VER [3] KERNEL_SZ [42027520] RAMDISK_SZ [34080769] OS_VERSION [12.0.0] OS_PATCH_LEVEL [2022-05] PAGESIZE [4096] CMDLINE [] + TEMPRC=0 + set +x OK, the new boot image "/data/local/tmp/img.568/new-boot.img" successfully created -rw-r--r-- 1 root root 100663296 2022-10-24 19:35 /data/local/tmp/img.568/new-boot.img Now rewriting "/dev/block/by-name/boot_b" using "/data/local/tmp/img.568/new-boot.img" ... 196608+0 records in 196608+0 records out 100663296 bytes (96 M) copied, 4.498630 s, 21 M/s TWRP successfully installed in "/dev/block/by-name/boot_b" ASUS_I006D:/ #

To install TWRP using the script without user input these steps are neccessary:

reboot the phone into the bootloader
boot the phone from the TWRP image that should be installed on the phone
copy the script install_twrp_from_within_twrp.sh to the phone
execute the script install_twrp_from_within_twrp.sh on the phone in an adb shell

To automate these steps I wrote a wrapper script :

install_twrp.sh

The wrapper script install_twrp.sh must be executed on a machine running Linux with the phone attached via USB.

The wrapper script will check the current status of the phone, boot the phone from the TWRP image, copy the script install_twrp_from_within_twrp.sh to the phone and execute it on the phone to install TWRP into either the active or the inactive boot partition.

The script install_twrp_from_within_twrp.sh must be in the same directory as install_twrp.sh (if not set the variable TWRP_INSTALL_SCRIPT before executing the script -- see the script usage below).

The usage for the script install_twrp.sh is:

[xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ ./install_twrp.sh -h ./install_twrp.sh version - 1.0.0.0 - install TWRP to the boot partition of a phone running Android using TWRP Usage install_twrp_via_twrp.sh [boot_slot] [wait=n] [dd|fastboot] [twrp_image] All parameter are optional The parameter "boot_slot" can be a, b, active, inactive, next, current; default is the current boot slot of the phone Use the parameter "dd" to request repatching via dd in an adb session ; use the parameter "fastboot" to request repatching via "fastboot" Default is to use "dd" to flash the patched boot image The value for the parameter "wait" is the number of seconds to wait before starting the script "install_twrp_from_within_twrp.sh" on the phone This seems to be necessary to avoid errors while repacking the boot image. The default wait time is 10 seconds. The parameter "twrp_image" can be used to define another TWRP image to use. The parameter is optional - the default for "twrp_image" is hardcoded in the script (variable TWRP_IMAGE) The default TWRP image of the script is the TWRP for the ASUS Zenfone 8. The phone to patch must be attached via USB. The phone can be either in fastboot mode, in normal mode with enabled adb support, or already booted from the TWRP image The script uses the script "install_twrp_from_within_twrp.sh" to install TWRP. The script install_twrp_from_within_twrp.sh must be in the same directory as this script. The script will be copied to the phone and then executed on the phone. Set the variable TWRP_INSTALL_SCRIPT to the name of the script to use before starting this script if another script should be used . To change some of the values used by the script you can set environment variables before starting the script: Set the environment variable REBOOT to "yes" before starting the script to automatically reboot the phone after patching the new image Set the environment variable REBOOT to "no" before starting the script to disable the automatic reboot after patching the new image. Set the environment variable SERIAL_NUMBER to the serial number of the phone to patch if there is more then one phone connected via USB Set the environment variable ADB_OPTIONS to the options to be used with the adb command Set the environment variable FASTBOOT_OPTIONS to the options to be used with the fastboot command Set the environment variable TMP_DIR_ON_THE_PHONE to the temporary directory to use on the phone The default TWRP image to use is "/data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D.img" The default script executed on the phone to install TWRP is "./install_twrp_from_within_twrp.sh" [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$

Without any parameter the script install_twrp.sh will install TWRP into the active boot partition.

Example for install_twrp.sh

[xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ ./install_twrp.sh ./install_twrp.sh version - 1.0.0.0 - install TWRP to the boot partition of a phone running Android using TWRP Using the TWRP image hardcoded in the script: "/data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D.img" Checking the script prerequisites ... Will wait 10 second(s) before starting the script on the phone The script is running as user "xtrnaw7" -- will use "sudo" for the fastboot commands ... Will patch the boot partition on the attached phone with the serial number "M6AIB760D0939LX" Using the options "-d -s M6AIB760D0939LX " for the adb commands Using the options " -s M6AIB760D0939LX " for the fastboot commands Checking for a connected phone booted into fastboot mode ... No attached phone in fastboot mode found Checking for an attached phone with working access via adb (USB) ... Linux localhost 5.4.147-Omni-qgki-perf-gea6f815f5af9-dirty #46 SMP PREEMPT Sat Oct 15 08:38:07 CEST 2022 aarch64 ... found a phone connected via USB with working adb access The phone is booted in normal mode Booting the phone into the fastboot mode now ... Waiting up to 60 seconds for the boot into the fastboot mode ... Booting the phone from the TWRP image "/data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D.img" now ... Sending 'boot.img' (98304 KB) OKAY [ 2.500s] Booting OKAY [ 10.440s] Finished. Total time: 12.983s Waiting up to 60 seconds for the boot of the phone .... Retrieving the current boot slot from the phone ... The current boot slot is "_a" The boot slot to patch is "_a" The boot partition to patch is "boot_a" Checking if the directory "/data/local/tmp" exists on the phone .... /data/local/tmp Copying the script "./install_twrp_from_within_twrp.sh" to the phone ... ./install_twrp_from_within_twrp.sh: 1 file pushed, 0 skipped. 216.6 MB/s (13141 bytes in 0.000s) Creating the boot image file "/data/local/tmp/boot_a.327964.img" from the partition "/dev/block/by-name/boot_a" ... 196608+0 records in 196608+0 records out 100663296 bytes (96 M) copied, 0.311720 s, 308 M/s Checking the result ... -rw-rw-rw- 1 root root 100663296 2022-10-24 18:02 /data/local/tmp/boot_a.327964.img The check sums are: The check sum of the boot partition "/dev/block/by-name/boot_a" on the phohe is "728455636" The check sum of the boot image file on the phone is "/data/local/tmp/boot_a.327964.img" is "728455636" Installing TWRP using the boot image file "/data/local/tmp/boot_a.327964.img" ... ---------------------------------------------------------------------- [2022.10.24 20:02:43] Waiting now 10 seconds ... + adb -d -s M6AIB760D0939LX shell /data/local/tmp/install_twrp_from_within_twrp.sh /data/local/tmp/boot_a.327964.img /data/local/tmp/boot_a_witn_twrp.327964.img Installing TWRP into a boot image or boot partition via script Checking the running OS ... OK, running a shell in TWRP: "twrp_I006D" version "3.7.0_12-0" Checking the prerequisites for installing TWRP ... Using the file "/data/local/tmp/boot_a.327964.img" as source for the installation of TWRP Creating the boot image with TWRP in the file "/data/local/tmp/boot_a_witn_twrp.327964.img" Checking if the required executables exist ... OK, the file "/system/bin/magiskboot" exists and is executable OK, the file "/system/bin/cpio" exists and is executable OK, the file "/system/bin/sha256sum" exists and is executable OK, the file "/system/bin/getprop" exists and is executable Checking if the requried data files exist ... OK, the file "/ramdisk-files.sha256sum" exists OK, the file "/ramdisk-files.txt" exists OK, the file "/data/local/tmp/boot_a.327964.img" exists Checking the check sums of the files for the new ramdisk ... OK, the check sums of the files for the new ramdisk are okay Directory "/data/local/tmp" successfully created Directory "/data/local/tmp/img.583" successfully created Directory "/data/local/tmp/img_files.583" successfully created The temporary directory to unpack the boot image is "/data/local/tmp/img_files.583" Unpacking the boot image from "/data/local/tmp/boot_a.327964.img" ... Parsing boot image: [/data/local/tmp/boot_a.327964.img] HEADER_VER [3] KERNEL_SZ [42025472] RAMDISK_SZ [34080743] OS_VERSION [12.0.0] OS_PATCH_LEVEL [2022-05] PAGESIZE [4096] CMDLINE [] KERNEL_FMT [raw] RAMDISK_FMT [gzip] VBMETA OK, "/data/local/tmp/boot_a.327964.img" successfully unpacked to "/data/local/tmp/img_files.583" : Creating a backup of the original ramdisk ... total 74412 -rw-rw-rw- 1 root root 50 2022-10-24 18:02 header -rw-r--r-- 1 root root 42025472 2022-10-24 18:02 kernel -rw-r--r-- 1 root root 34080743 2022-10-24 18:02 ramdisk.cpio.org Creating the new ramdisk with TWRP ... New ramdisk with TWRP "/data/local/tmp/img_files.583/ramdisk.cpio" successfully created. -rw-rw-rw- 1 root root 85852380 2022-10-24 18:02 /data/local/tmp/img_files.583/ramdisk.cpio -rw-r--r-- 1 root root 34080743 2022-10-24 18:02 /data/local/tmp/img_files.583/ramdisk.cpio.org Repacking the boot image into the file "/data/local/tmp/boot_a_witn_twrp.327964.img" ... + cd /data/local/tmp/img_files.583 + /system/bin/magiskboot repack /data/local/tmp/boot_a.327964.img /data/local/tmp/boot_a_witn_twrp.327964.img Parsing boot image: [/data/local/tmp/boot_a.327964.img] HEADER_VER [3] KERNEL_SZ [42025472] RAMDISK_SZ [34080743] OS_VERSION [12.0.0] OS_PATCH_LEVEL [2022-05] PAGESIZE [4096] CMDLINE [] KERNEL_FMT [raw] RAMDISK_FMT [gzip] VBMETA Repack to boot image: [/data/local/tmp/boot_a_witn_twrp.327964.img] HEADER_VER [3] KERNEL_SZ [42025472] RAMDISK_SZ [34080758] OS_VERSION [12.0.0] OS_PATCH_LEVEL [2022-05] PAGESIZE [4096] CMDLINE [] + TEMPRC=0 + set +x OK, the new boot image "/data/local/tmp/boot_a_witn_twrp.327964.img" successfully created -rw-r--r-- 1 root root 100663296 2022-10-24 18:03 /data/local/tmp/boot_a_witn_twrp.327964.img Note: Flashing the new image to the boot partition was not requested + set +x ---------------------------------------------------------------------- Checking the result ... The patched boot image is "/data/local/tmp/boot_a_witn_twrp.327964.img" -rw-r--r-- 1 root root 100663296 2022-10-24 18:03 /data/local/tmp/boot_a_witn_twrp.327964.img [2022.10.24 20:03:05] Waiting now 5 seconds ... Patching the partition "/dev/block/by-name/boot_a" from the patched boot image file "/data/local/tmp/boot_a_witn_twrp.327964.img" via dd ... 196608+0 records in 196608+0 records out 100663296 bytes (96 M) copied, 4.865411 s, 20 M/s Checking the result .... The check sums for the images and devices on the phone are: Checksum Size File/Device name ------------------------------------- 728455636 100663296 /data/local/tmp/boot_a.327964.img 2360878166 100663296 /data/local/tmp/boot_a_witn_twrp.327964.img 2360878166 100663296 /dev/block/by-name/boot_a OK, patching the boot partition "/dev/block/by-name/boot_a" was successfull *** Press return to reboot the phone now Rebooting the phone now ... [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$

Advanced Usage

To install TWRP and Magisk via script these commands can be used :

# install the Magisk App (if not already done) # cat Magisk-25.1_25100.apk | adb shell pm install -S $( ls -l Magisk-25.1_25100.apk | awk '{ print $5}' ) # install TWRP and Magisk into the boot partition # REBOOT=no ./install_twrp.sh ; REBOOT=yes ./install_magisk_via_twrp.sh

To install a new OS image including TWRP and Magisk these commands can be used:

# boot the phone into the boot loader # adb reboot bootloader # boot the phone from the TWRP image # sudo fastboot boot /data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D.img # copy the OS image to install to the phone # adb push /data/develop/android/OmniROM/out/target/product/zenfone8/omni-12-20221015-zenfone8-MICROG.zip /sdcard/Download/ # install the OS image # adb shell twrp install /sdcard/Download/omni-12-20221015-zenfone8-MICROG.zip # reboot into the new OS # adb shell twrp reboot # install the Magisk app # cat Magisk-v25.2.apk | adb shell pm install -S $( ls -l Magisk-v25.2.apk | awk '{ print $5}' ) # install TWRP and Magisk # REBOOT=no ./install_twrp.sh ; REBOOT=yes ./install_magisk_via_twrp.sh
The documentation for the script install_magisk_via_twrp.sh can be found here:

How to install Magisk into the boot partition using a script.

Details

The script install_twrp_from_within_twrp.sh was tested on an ASUS Zenfone 8 with

OmniROM 12 (Android 12) and TWRP 3.7.0.12
OmniROM 12 (Android 12) and TWRP 3.6.1.12
Original Android 12 from ASUS and TWRP 3.7.0.12
OmniROM 13 (Android 13) and TWRP 3.7.0.12 (see note below)

Important Update 04.12.2022 /bs

In TWRP installed in the boot partition of an ASUS Zenfone 8 running early versions of OmniROM 13 the touch screen does not work. You can only use TWRP in an adb session in this configuration.
This bug was fixed in OmniROM 13 from 04.12.2022.

To get around this bug boot the phone from the image file with TWRP, e.g.:

sudo fastboot boot twrp-3.7.0_12-0-I006D.img
or, better, update OmniROM 13 to the image from 04.12.2022 or later.

The script install_twrp_from_within_twrp.sh does not create a backup of the boot partition before rewriting it.

The script install_twrp_from_within_twrp.sh creates temporary files in the directory ${TMPDIR} on the phone; the default value for this variable in the script is

TMPDIR="${TMPDIR:=/cache/install_twrp.$$}"

The default value for the variable in the tested TWRP versions is /data/local/tmp so without changes the temporary files will be created in that directory.
The script does not delete the temporary files so a manual cleanup of the directory might be necessary .

Trouble Shooting

For unknown reasons repacking the boot image fails sometimes and the script ends with an error message like this:

ERROR: The size of the new boot image file is 56713216 but the size of the original image file is 100663296 -- that does not match (RC=33)

To avoid this error increase the wait time using the script parameter "wait=n" (the default wait time is 10 seconds).

If increasing the wait time does not help just restart the wrapper script install_twrp.sh again while the phone is still booted from the TWRP image. So a workaround for this problem is :

install_twrp.sh ; [ $? = 33 ] && install_twrp.sh

The following error message appears when executing the script install_twrp_from_within_twrp.sh after booting the already installed TWRP:

ERROR: Error checking the check sums of the files for the new ramdisk ./system/etc/fstab.postinstall: FAILED

Installing TWRP via script only works if the phone is booted from the TWRP image.
To ignore this error the parameter -f can be used - but this is NOT recommended.

How to add additional files to an TWRP image

URL: https://xdaforums.com/t/how-to-add-additional-files-to-an-twrp-image.4511827/

How to add additional files to an TWRP image

The recovery image TWRP is very useful for the maintenance of the Android OS.

It's the Swiss Army knife for maintaining phones running the Android OS but sometimes you might miss some files in the TWRP recovery.

To add additional files to an TWRP image the instructions from the HowTo How to change files in the boot image can be used.

To make sure that the new files are also added to an TWRP installed in the boot partition additional steps are necessary.

To unpack and repack the TWRP image the Magisk binary magiskboot is necessary. This binary is part of TWRP so creating the new image should be done while the phone is booted from the TWRP image.

So to add additional files to an TWRP image do:
# boot into the TWRP image # adb reboot bootloader sudo fastboot boot /data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D.img

Wait until booting the TWRP image is done and open an adb shell

adb shell

Create some directories for the work on the phone:

mkdir -p /data/develop/workdir/twrp/ramdisk

Copy the TWRP image file from your PC to the working directory on the phone

adb push twrp-3.7.0_12-0-I006D.img /data/develop/workdir/twrp

Unpack the image file:

cd /data/develop/workdir/twrp          magiskboot unpack -h twrp-3.7.0_12-0-I006D.img

e.g.

ASUS_I006D:/data/develop/workdir/twrp # magiskboot unpack -h twrp-3.7.0_12-0-I006D.img Parsing boot image: [twrp-3.7.0_12-0-I006D.img] HEADER_VER      [3] KERNEL_SZ       [18627321] RAMDISK_SZ      [34811448] OS_VERSION      [99.87.36] OS_PATCH_LEVEL [2099-12] PAGESIZE        [4096] CMDLINE         [twrpfastboot=1] KERNEL_FMT      [gzip] RAMDISK_FMT     [gzip] VBMETA ASUS_I006D:/data/develop/workdir/twrp # ASUS_I006D:/data/develop/workdir/twrp # ls -ltr total 324223 -rw-rw---- 1 u0_a119 media_rw 100663296 2022-10-12 13:38 twrp-3.7.0_12-0-I006D.img -rw-r--r-- 1 root    root     100663296 2022-10-28 15:55 new-boot.img -rw-rw-rw- 1 root    root            66 2022-10-28 16:14 header -rw-r--r-- 1 root    root      41943552 2022-10-28 16:14 kernel -rw-r--r-- 1 root    root      88384000 2022-10-28 16:14 ramdisk.cpio drwxrwxrwx 2 root    root          3452 2022-10-28 16:15 ramdisk ASUS_I006D:/data/develop/workdir/twrp #

Unpack the ramdisk from the TWRP image:

cd /data/develop/workdir/twrp/ramdisk          cpio -idm <../ramdisk.cpio

The directory /data/develop/workdir/twrp/ramdisk now contains the contents of the ramdisk used for TWRP.

Now add the new files for the image to the directory /data/develop/workdir/twrp/ramdisk:

In this example we will add the binaries zsh, sqlite3, and vi.

cd /data/develop/workdir/twrp/ramdisk cp ../../newfiles/sqlite3 ./system/bin/ cp ../../newfiles/zsh ./system/bin/ cp ../../newfiles/vi ./system/bin/

Note:

In case you want to add new binaries :

To check if the binaries will work in TWRP execute the binaries while the phone is booted from the original TWRP image. If the binaries need additional libraries not included in TWRP just also add them to the TWRP image into the directory ./system/lib64 of the image.

The next step is necessary to make sure that TWRP will copy the new files also to a TWRP installation in a boot partition:

For each new file add an entry to the files

./ramdisk-files.sha256sum
./ramdisk-files.txt

in the directory with the contents from the ramdisk.

The file ramdisk-files.txt contains the name and path of the files and directories used in the ramdisk for the boot partition and the file ramdisk-files.sha256sum contains the checksum for each of the files for the ramdisk.

Notes:

[Update 26.11.2023/bs] New directories for the image must also be added to the file ramdisk-files.txt.

If you change an existing file that is mentioned in the file ./ramdisk-files.sha256sum you must also correct the entry in that file.

So do for each new file:
echo <file_with_path_not_starting_with_a_dot_or_slash> >>ramdisk-files.txt ; done sh256sum <file_with_path_startinge with_a_slash> >>ramdisk-files.sha256sum

e.g:

ASUS_I006D:/data/develop/workdir/twrp/ramdisk # for i in system/bin/sqlite3 system/bin/vi system/bin/zsh ; do sha256sum ./$i >>ramdisk-files.sha256sum ; echo "$i" >>ramdisk-files.txt ; done ASUS_I006D:/data/develop/workdir/twrp/ramdisk #

Check the result.

ASUS_I006D:/data/develop/workdir/twrp/ramdisk # tail -4 ramdisk-files.sha256sum 8a8ff76cbda445f08ba3552598085089883f9d594cd7716e90f3411a8b82f8a0 ./twres/fonts/DroidSansMono.ttf 09fcec146d9ecba0f2a0b7362b52d2f87607a3c5760c94ba5fcd900e423e16f8 ./system/bin/sqlite3 3a221d10ffc275cadaf79fb1a00c39c9083ce672ea111205301e277091a04b51 ./system/bin/vi 354e415da42503ed4211907adf93ed8bfb14ce2e2d8418fcd7a3227873b4b53f ./system/bin/zsh ASUS_I006D:/data/develop/workdir/twrp/ramdisk # ASUS_I006D:/data/develop/workdir/twrp/ramdisk # tail -4 ramdisk-files.txt twres/fonts/DroidSansMono.ttf system/bin/sqlite3 system/bin/vi system/bin/zsh ASUS_I006D:/data/develop/workdir/twrp/ramdisk #

Correct the entry for the file ramdisk-files.txt in the file ramdisk-files.sha256sum :
cd /data/develop/workdir/twrp/ramdisk cp ./ramdisk-files.sha256sum ./ramdisk-files.sha256sum.org NEW_ENTRY="$( sha256sum ./ramdisk-files.txt )" && sed -i -e "s#.*\./ramdisk-files.txt#${NEW_ENTRY}#g" ./ramdisk-files.sha256sum

Check the entries in the file ramdisk-files.sha256sum

sha256sum --status -c ./ramdisk-files.sha256sum ; echo $?

If the output of that command is not 0 use

sha256sum -c ./ramdisk-files.sha256sum   | grep -v OK

to get the entry with the invalid checksum. Fix the entry and check again.


Now repack the ramdisk:

cd /data/develop/workdir/twrp/ramdisk find . | cpio -o >../ramdisk.cpio

And recreate the image file:

cd /data/develop/workdir/twrp magiskboot repack twrp-3.7.0_12-0-I006D.img
e.g

ASUS_I006D:/data/develop/workdir/twrp # magiskboot repack twrp-3.7.0_12-0-I006D.img Parsing boot image: [twrp-3.7.0_12-0-I006D.img] HEADER_VER      [3] KERNEL_SZ       [18627321] RAMDISK_SZ      [34811448] OS_VERSION      [99.87.36] OS_PATCH_LEVEL [2099-12] PAGESIZE        [4096] CMDLINE         [twrpfastboot=1] KERNEL_FMT      [gzip] RAMDISK_FMT     [gzip] VBMETA Repack to boot image: [new-boot.img] HEADER_VER      [3] KERNEL_SZ       [18618540] RAMDISK_SZ      [37358888] OS_VERSION      [99.87.36] OS_PATCH_LEVEL [2099-12] PAGESIZE        [4096] CMDLINE         [twrpfastboot=1] ASUS_I006D:/data/develop/workdir/twrp #

The file new-boot.img contains the TWRP image with the additional files.:

ASUS_I006D:/data/develop/workdir/twrp # ls -ltr new-boot.img -rw-r--r-- 1 root    root     100663296 2022-10-28 16:28 new-boot.img ASUS_I006D:/data/develop/workdir/twrp #

To test the image, rename the file, copy the file to an PC and boot the phone from the image:

mv new-boot.img twrp-3.7.0_12-0-I006D-enhanced.img

Execute on the PC:

adb pull /data/develop/workdir/twrp/twrp-3.7.0_12-0-I006D-enhanced.img adb reboot bootloader sudo fastboot boot twrp-3.7.0_12-0-I006D-enhanced.img

Connect via adb to the phone booted from the new TWRP image and check the result:

ASUS_I006D:/ # ls -l /system/bin/zsh /system/bin/sqlite3 /system/bin/vi -rwxr-xr-x 1 root root 1714720 2022-10-28 16:19 /system/bin/sqlite3 -rwxr-xr-x 1 root root 1815152 2022-10-28 16:19 /system/bin/vi -rwxr-xr-x 1 root root 2004808 2022-10-28 16:19 /system/bin/zsh ASUS_I006D:/ # ASUS_I006D:/ # /system/bin/sqlite3 --version 3.39.3 2022-09-05 11:02:23 4635f4a69c8c2a8df242b384a992aea71224e39a2ccab42d8c0b0602f1e826e8 ASUS_I006D:/ # ASUS_I006D:/ # /system/bin/vi --version | head -1 VIM - Vi IMproved 8.1 (2018 May 18) ASUS_I006D:/ # ASUS_I006D:/ # /system/bin/zsh --version zsh 5.9 (aarch64-unknown-linux-android) ASUS_I006D:/ #

Advanced Usage

To be able to distinguish the new image from the original one you can change or add values in the file ./default.prop in the ramdisk for the image, e.g.:

# before creating the new ramdisk

cd /data/develop/workdir/twrp/ramdisk echo "ro.product.type=enhanced" >>default.prop

Note:

The file default.prop is not listed in the checksum file ramdisk-files.sha256sum for the image.

Then when booted from the image you can check the new property, e.g.:

ASUS_I006D:/ # getprop ro.product.type enhanced ASUS_I006D:/ #

Notes:

Adding new files to ramdisk-files.sha256sum is not mandatory but recommended

To add files to the image that should not be copied to an TWRP installation in a boot partition just add them to the ramdisk but not to the file ramdisk-files.txt.

see also How to install TWRP via script

History of this entry

Udpate 28.11.2022 /bs

see also How to compile TWRP for the ASUS Zenfone 8

Update 26.11.2023 /bs

New directories for the image must also be added to the file ramdisk-files.txt.

Update 11.01.2024 /bs

If booting the phone from the new TWRP image ends in a ramdump or an endless loop:

It's important to use the correct version of the magiskboot binary to unpack and repack the image file. The magiskboot that is part of the TWRP image is always the correct version so I strongly recommend to create the new image file while the phone is booted from the TWRP image that should be extended.

How to process Android boot image files on a PC running the Linux OS

URL: https://xdaforums.com/t/how-to-process-android-boot-image-files-on-a-pc-running-the-linux-os.4513845/

How to process Android boot image files on a PC running the Linux OS

To unpack and repack the boot image for phones running the Android OS the Magisk binary magiskboot is necessary (well, there might be other tools for this task but magiskboot seems to be one of the best tools for this task). Therefor these kind of tasks will usually be done on a phone running the Android OS.

But there is also a magiskboot binary for x86 CPUs in the Magisk apk file and that executable can be used to work with boot images for Android phones on a PC running the Linux OS on a machine with X86 CPU. To retrieve the x86 version of magiskboot from the Magisk apk file do:

# to use the 32 bit version # unzip Magisk-v25.2.apk lib/x86/libmagiskboot.so mv lib/x86/libmagiskboot.so ./magiskboot chmod 755 ./magiskboot

or

# to use the 64 bit version # unzip Magisk-v25.2.apk lib/x86_64/libmagiskboot.so mv lib/x86_x64/libmagiskboot.so ./magiskboot chmod 755 ./magiskboot

Example:

[xtrnaw7@t15g /data/develop/android/magisk]$ unzip Magisk-v25.2.apk lib/x86_64/libmagiskboot.so Archive: Magisk-v25.2.apk version=25.2 versionCode=25200 inflating: lib/x86_64/libmagiskboot.so [xtrnaw7@t15g /data/develop/android/magisk]$ [xtrnaw7@t15g /data/develop/android/magisk]$ mv lib/x86_64/libmagiskboot.so ./magiskboot [xtrnaw7@t15g /data/develop/android/magisk]$ [xtrnaw7@t15g /data/develop/android/magisk]$ chmod 755 magiskboot [xtrnaw7@t15g /data/develop/android/magisk]$ [xtrnaw7@t15g /data/develop/android/magisk]$ file ./magiskboot ./magiskboot: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=b2d42d7bab2f70a0d66fc606a1fb2726a0559fed, stripped [xtrnaw7@t15g /data/develop/android/magisk]$ [xtrnaw7@t15g /data/develop/android/magisk]$ ldd ./magiskboot not a dynamic executable [xtrnaw7@t15g /data/develop/android/magisk]$ [xtrnaw7@t15g /data/develop/android/magisk]$ ./magiskboot MagiskBoot - Boot Image Modification Tool Usage: ./magiskboot <action> [args...] Supported actions: ... [xtrnaw7@t15g /data/develop/android/magisk]$
The binary magiskboot from the Magisk apk file is a statically linked executable for 32 bit or 64 bit. Therefor it should run on any Linux distribution that supports executing x86 executables . I tested the binary from the apk file with Magisk v25.2 on these distributions:

Fedora 36 (64 Bit, kernel 6.x)
Redhat 8.6 (64 Bit, kernel 4.x)
Redhat 6.10 (64 Bit, kernel 2.x)
GRML (32 Bit, kernel 5.x, the 32 bit version only)
RockyLinux 9.0 (64 Bit, kernel 5.x)
openSUSE LEAP 15.2 (64 Bit, kernel 5.x)

Unpacking a boot image file using magiskboot on a PC running the Linux OS can be done using this command;

./magiskboot unpack boot_a.img

To change files in the ramdisk from the boot image use the magiskboot parameter cpio :

[xtrnaw7@t15g /data/develop/android/test2]$ ./magiskboot
MagiskBoot - Boot Image Modification Tool

Usage: ./magiskboot <action> [args...]

Supported actions:
...

cpio <incpio> [commands...]
Do cpio commands to <incpio> (modifications are done in-place)
Each command is a single argument, add quotes for each command
Supported commands:
exists ENTRY
Return 0 if ENTRY exists, else return 1
rm [-r] ENTRY
Remove ENTRY, specify [-r] to remove recursively
mkdir MODE ENTRY
Create directory ENTRY in permissions MODE
ln TARGET ENTRY
Create a symlink to TARGET with the name ENTRY
mv SOURCE DEST
Move SOURCE to DEST
add MODE ENTRY INFILE
Add INFILE as ENTRY in permissions MODE; replaces ENTRY if exists
extract [ENTRY OUT]
Extract ENTRY to OUT, or extract all entries to current directory
test
Test the current cpio's status
Return value is 0 or bitwise or-ed of following values:
0x1:Magisk 0x2:unsupported 0x4:Sony
patch
Apply ramdisk patches
Configure with env variables: KEEPVERITY KEEPFORCEENCRYPT
backup ORIG
Create ramdisk backups from ORIG
restore
Restore ramdisk from ramdisk backup stored within incpio
sha1
Print stock boot SHA1 if previously backed up in ramdisk
...

[xtrnaw7@t15g /data/develop/android/test2]$

E.g. to change the file system/etc/ramdisk/build.prop in the ramdisk, extract the file from the ramdisk

./magiskboot cpio ramdisk.cpio "extract system/etc/ramdisk/build.prop build.prop"

edit the file, and add the file back to the ramdisk using this command:

./magiskboot cpio ramdisk.cpio "add 0644 system/etc/ramdisk/build.prop build.prop"

Notes

Note that I did not yet manage to create a working boot image with magiskboot running in the Linux OS after extracting, changing, and recreating the ramdisk via the Linux cpio command (as Linux user root, of course):

Creating a new boot image this way with magiskboot running in the LInux OS works without error but booting the new created boot image always ends with a ramdump error on the phone. So extracting the ramdisk via Linux cpio command can only be used to view the files in a boot image file for an Android Phone from wthin the Linux OS.

Trouble Shooting

To test the new boot image without installing it use this approach:

# boot the phone into the bootloader # adb reboot bootloader # boot the phone from the patched image file # sudo fastboot boot ./new-boot.img
If booting the phone works and everything is fine flash the image to the active boot partition

In case you get a ramdump error after booting the phone from the new boot partition just restart the phone into the bootloader mode and flash the old image back to the boot partition.

How to make files in /system writable

URL: https://xdaforums.com/t/how-to-make-files-in-system-writable.4514603/

How to make files in /system writable

In Android 12 /system is mounted read-only can not be remounted read-write anymore.

Sometimes it's useful that one or more files in /system are writable (for example for develop tasks or for testing)

This can be implemented using Magisk (see How to change files in the directory /system for more details)

Example :

Make the file /system/etc/vimrc writable

# create a dummy Magisk module # mkdir -p /data/adb/modules/writable_system/system/etc # copy the file that should be writable to the Magisk module directory # cp /system/etc/vimrc /data/adb/modules/writable_system/system/etc/ # make the file in the Magisk module directory writable # chmod +w /data/adb/modules/writable_system/system/etc/vimrc

Now reboot the phone.

After the reboot the file /system/etc/vimrc is writable by the user root, Example:

ASUS_I006D:/ # id uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 ASUS_I006D:/ # ASUS_I006D:/ # ls -l /system/etc/vimrc -rw-r--r-- 0 root root 3350 2022-11-04 11:36 /system/etc/vimrc ASUS_I006D:/ # tail -2 /system/etc/vimrc         \ | wincmd p | diffthis endif ASUS_I006D:/ # ASUS_I006D:/ # echo '" Test Comment' >>/system/etc/vimrc ASUS_I006D:/ # ASUS_I006D:/ # tail -2 /system/etc/vimrc endif " Test Comment ASUS_I006D:/ #
Only the user root can access the directory /data/adb. Therefor the files configured using this approach are only writable by the user root.

To make a file in /system writable for non-root users use this method:

Open a (adb) shell and execute as user shell:

# # create a directory that is writable for the user shell # mkdir /data/local/tmp/writable_system mkdir /data/local/tmp/writable_system/etc # # copy the file that should be writable to that directory # cp /system/etc/vimrc /data/local/tmp/writable_system/etc
The next commands must be executed as user root:

# create dummy Magisk module # mkdir -p /data/adb/modules/writable_system/system/etc # # create a symbolic link to the file in the writable directory in the directory with the dummy Magisk module # ln -s /data/local/tmp/writable_system/etc/vimrc /data/adb/modules/writable_system/system/etc
Now reboot the phone.

After the reboot the file /system/etc/vimrc is writable by the user shell, Example:

ASUS_I006D:/ $ id uid=2000(shell) gid=2000(shell) groups=2000(shell),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),1078(ext_data_rw),1079(ext_obb_rw),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats),3009(readproc),3011(uhid),3012(readtracefs) context=u:r:shell:s0 ASUS_I006D:/ $ ASUS_I006D:/ $ tail -2 /system/etc/vimrc         \ | wincmd p | diffthis endif ASUS_I006D:/ $ ASUS_I006D:/ $ echo '" Test Comment' >>/system/etc/vimrc ASUS_I006D:/ $ ASUS_I006D:/ $ tail -2 /system/etc/vimrc endif " Test Comment ASUS_I006D:/ $

Important:

The writable directory can also be in a sub directory in /sdcard. But be aware that /sdcard is mounted late in the boot process so it might be that the overwritten file in /system will be used by the OS when the bind mount points to a non-existent file if using a sub directory in /sdcard.

The changes to the file done using these methods are "persistent" as long as Magisk is installed in the boot partition.

To restore the file with the original contents after each new reboot of the phone without removing the writable config open a (adb) shell as user root and execute:

# # restore the file /data/adb/modules/writable_system/system/etc/vimrc from the original file /system/etc/vimrc # # this must be done before Magisk creates the bind mounts # echo "cp /system/etc/vimrc /data/adb/modules/writable_system/system/etc/vimrc">/data/adb/post-fs-data.d/restore_vimrc.sh chmod 755 /data/adb/post-fs-data.d/restore_vimrc.sh

Now the file in the dummy Magisk module will be restored with the contents of the original file from /system after each reboot

To temporary access the original file from /system just stop the Magisk daemon, Example:

ASUS_I006D:/ # echo '"Test Test' >>/etc/vimrc ASUS_I006D:/ # ASUS_I006D:/ # tail -1 /etc/vimrc "Test Test ASUS_I006D:/ # ASUS_I006D:/ # id    uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 ASUS_I006D:/ # ASUS_I006D:/ # magisk --stop ASUS_I006D:/ # ASUS_I006D:/ # id uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 ASUS_I006D:/ # ASUS_I006D:/ # tail -1 /etc/vimrc endif ASUS_I006D:/ #

Note

Stopping the Magisk daemon will disable all bind mounts done by Magisk.
Restarting the Magisk daemon will not re-create the bind mount - to re-activate the bind mount for the writable file after stopping the Magisk daemon the phone must be rebooted.

To make more then one file writable in a sub directory in /system you can also replace the complete folder using these commands as user root:

# # make all files in /system/etc writable by the user root # mkdir -p /data/adb/modules/writable_system/system/etc cd /system/etc find . | cpio -pdum /data/adb/modules/writable_system/system/etc/ touch /data/adb/modules/writable_system/system/etc/.replace
Now Magisk will replace the directory /system/etc with the directory /data/adb/modules/writable_system/system/etc after the next reboot

Notes

You should test these commands with a not important file like /system/etc/vimrc before changing important files.

It is NOT recommended to use this approach on productive phones.

See How to change any file or directory using Magisk for another approach to change files on read-only mounted filesystems.

Trouble Shooting

As always: If something does not work like expected check the Magisk log file /cache/magisk.log and also check the infos in this post.

How to boot a phone from a TWRP recovery image via script

URL: https://xdaforums.com/t/how-to-boot-a-phone-from-a-twrp-recovery-image-via-script.4527547/

How to boot a phone from a TWRP recovery image via script

For installing and configuring an Android phone via script it is often required to boot the phone from an TWRP recovery image.

Because of this I wrote a Linux shell script to reboot an Android phone from an TWRP recovery image:

boot_phone_from_twrp.sh

The script boot_phone_from_trwp.sh detects the current status of the attached phone and uses the approbiate commands (fastboot and/or adb) to reboot the phone from the TWRP recovery image.

The usage for the script is:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./boot_phone_from_twrp.sh -h boot_phone_from_twrp.sh [-h|help|-H] [serial=#sn|s=#sn] [wait=n] [password=password] [decrypt] [usb_reset|no_usb_reset] [force|noforce] [reboot|noreboot] [checkonly|check] [reset_usb_only] [status] [twrp|Android|android|recovery|bootloader|sideload|fastboot|safemode|twrp_image_file] The returncode is 110 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $Use the parameter -H to get a detailed usage help:

detailed usage help for boot_phone_from_twrp.sh

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ date Sun Jul 14 01:02:53 PM CEST 2024 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./boot_phone_from_twrp.sh -H Usage boot_phone_from_twrp.sh [-h|help|-H] [serial=#sn|s=#sn] [wait=n] [password=password] [decrypt] [usb_reset|no_usb_reset] [force|noforce] [reboot|noreboot] [checkonly|check] [reset_usb_only] [status] [twrp|Android|android|recovery|bootloader|sideload|fastboot|safemode|twrp_image_file] All parameter are optional; without a parameter the script boots the phone from the TWRP image. Use the parameter "help" or "-H" to print the detailed usage help; use the parameter "-h" to print only the short usage help Use the parameter "serial=#sn" to define the serialnumber of the phone to process. This parameter is only necessary if there are more then one phones connected via USB and the environment variable SERIAL_NUMBER is not set. The value for the parameter "wait" is the maximum number of seconds to wait after booting from the TWRP image until the adb deamon is ready to use. The default value is 10 seconds. Use the parameter "password=passwd" to define the password used for the data partition on the phone. Use the parameter "force" to reboot the phone from the TWRP image even if it's booted from a TWRP installed in the boot or recovery partition. Use the parameter "noforce" to disable rebooting the phone from the TWRP image if it's booted from a TWRP installed in the boot or recovery partition. This is the default setting. Use the parameter "reboot" to reboot the phone from the TWRP image even if it's already booted from an TWRP image. Use the parameter "noreboot" to disable rebooting the phone from the TWRP image if it's already booted from an TWRP image This is the default setting. Use the parameter "checkonly" or "check" to only retrieve the current boot mode of the phone; the script returns the current status via return code and writes no messages; see below for the list of defined return codes. Use the parameter "status" to only print the current boot mode of the phone; the script also returns the current status via return code; see below for the list of defined return codes. Use the parameter "decrypt" to only decrypt the data partition. Use the parameter "no_usb_reset" to disable the reset of the USB port; the default is to reset the USB port if necessary if the executable usbreset is available via PATH Use the parameter "usb_reset" to force a reset of the USB port used for the phone if there is no access to the phone The reset of the USB port can only be done if executable "usbreset" is available via PATH variable Use the parameter "reset_usb_only" to only reset the USB port ussed for the phone The parameter "recovery", "bootloader", "sideload", and "fastboot" can be used to boot the phone in the specified mode. The parameter "Android" or "android" can be used to boot the phone into the Android OS installed on the phone. The parameter "safemode" can be used to boot the Android OS on the phone into the "safe mode". Note that this only works if the phone is already booted into the Android OS. The parameter "twrp_image_file" can be used to define the TWRP image to be used (every parameter that is not in the list of known parameter above is treated as TWRP image file) Environment variables overwrite the values defined in the script; script parameter overwrite the values defined in environment variables. The order used by the script to select the TWRP image to use is as follows: 1. If there is TRWP image in the parameter for the script, this TWRP image is used 2. If there is a TWRP image defined in the variable TWRP_IMAGE, this TWRP image is used 3. If the script can access the phone via adb; it tries to find the TWRP image necessary for the running OS 4. If there is still no TWRP image to use found, the TWRP image defined in the variable DEFAULT_TWRP_IMAGE hardcoded in the script will be used The TWPR image defined in the variable DEFAULT_TWRP_IMAGE is the TWRP image for the ASUS Zenfone 8 running the ASUS Android OS Returncodes: 0 - the phone was successfully booted from the TWRP image 1 - the phone is already booted from the TWRP image 2 - the phone is booted from TWRP installed in the boot or recovery partition 3 - the phone is booted into the Android OS 4 - the phone is booted into bootloader 5 - the phone is booted into the fastbootd 6 - the phone is booted into the safe mode of the Android OS 7 - the phone is booted into the a non-TWRP recovery installed in the boot or recovery partition 8 - the phone is booted into sideload mode 9 - the phone is booted into a recovery without working adb shell 100 - invalid parameter found 101 - TWRP image not found 102 - booting the phone into the bootloader failed 103 - booting the phone into the TWRP failed 104 - Booting into safemode is only possible if the phone is booted into the Android OS 105 - Decyrptig the phone is only possible if the phone is booted from TWRP 108 - Booting the phone into the requested mode failed 109 - too many phones connected 110 - usage help printed and exited 111 - found no TWRP image for the running OS 112 - One or more errors in the prereq found 113 - Booting the phone into the safe mode requires root access to the phone 114 - no known boot method for the current status of the phone 252 - access to the phone failed 253 - requirement check failed (e.g. one or more required executables not found, etc) 254 - unknown error The phone to boot must be attached via USB. The phone can be either in fastboot mode, in normal mode with enabled adb access, in the boot loader, already booted from an installed TWRP or an TWRP image, or booted into another recovery like the LineageOS recovery with enabled adb access. Phones in EDL mode are also detected; but the script can nothing do with a phone in EDL mode. To change some of the values used by the script you can set environment variables before starting the script: Set the environment variable TWRP_IMAGE to the name of the TWRP image file that should be used (the parameter of the script overwrites the variable) Set the environment variable SERIAL_NUMBER to the serial number of the phone to patch if there is more then one phone connected via USB Set the environment variable ADB_OPTIONS to the options to be used with the adb command Set the environment variable FASTBOOT_OPTIONS to the options to be used with the fastboot command Set the environment variable FASTBOOT_TIMEOUT to the maximum number of seconds to wait for a fastboot command to finish (default are 3 seconds) Set the environment variable FASTBOOT_WAIT_TIME to the maximum number of seconds to wait after booting the phone from the bootloader (default are 60 seconds) Set the environment variable ADB_BOOT_WAIT_TIME to the maximum number of seconds to wait after booting the phone until there is a working adb connection (default are 120 seconds) Set the environment variable ADB_DAEMON_WAIT_TIME to the maximum number of seconds to wait for the restart of the adb daemon after booting into TWRP Set the environment variable DECRYPT_DATA_WAIT_TIME to the maximum number of seconds to wait after booting the phone until /data is decrypted (default are 150 seconds) Set the environment variable ADB_SERVICE_WAIT_TIME to the maximum number of seconds to wait for the Android service "package" got ready after booting into the Android OS (default are 30 seconds) Set the environment variable ADB to the adb executable that should be used; default: search adb in the PATH Set the environment variable FASTBOOT to the fastboot executable that should be used; default: search fastboot in the PATH Set the environment variable TIMEOUT to the timeout executable that should be used; default: search timeout in the PATH Set the environment variable USBRESET to the usbreset executable that should be used if the parameter usb_reset is used; default: search timeout in the PATH Set the environment variable CHECK_ONLY to 0 (= ${__TRUE}) to only check the status of the phone Set the environment variable PRINT_STATUS and also the variable CHECK_ONLY to 0 = ${__TRUE} to only print the status of the phone Set the environment variable RESET_THE_USB_PORT to 0 = ${__TRUE} to force a reset of the USB port used by the phone if the access is not working Set the environment variable USER_PASSWORD to the password used for the data partition (see here: https://twrp.me/faq/openrecoveryscript.html for the format of the password) The default TWRP image to use is "/data/backup/ASUS_ZENFONE8/twrp/current_twrp.img" The returncode is 110 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

When more than one phone is connected via USB use the parameter serial=#sn or the environment variable SERIAL_NUMBER to define the serial number of the phone to use.

To only retrieve the current status of the attached phone execute the script with the parameter checkonly :

./boot_phone_from_twrp.sh checkonly

The script then only checks the status of the phone and returns the current status in the return code:

     1 - the phone is booted from the TWRP image
     2 - the phone is booted from TWRP installed in the boot or recovery partition
     3 - the phone is booted into the Android OS
     4 - the phone is booted into bootloader
     5 - the phone is booted into the fastbootd
     6 - the phone is booted into the safe mode of the Android OS
     7 - the phone is booted into the LineageOS recovery installed in the boot or recovery partition
     8 - the phone is booted into sideload mode
     9 - the phone is booted into a recovery without working adb shell

    10 - error retrieving the status of the attached phone (or no phone connected)
    11 - the phone is attached but without permissions to access via fastboot or adb
    12 - the phone seems to be in EDL mode

If the parameter status is used the script also prints a message with the current status of the phone, examples:

./boot_phone_from_twrp.sh checkonly|status

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./boot_phone_from_twrp.sh checkonly ; echo "RC is $?" RC is 3 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./boot_phone_from_twrp.sh status ; echo "RC is $?" boot_phone_from_twrp.sh v3.0.3.0 - boot a phone from a TWRP image The script is running as user "xtrnaw7" -- using "sudo" for the fastboot commands ... Using the attached phone with the serial number "M6AIB760D0939LX" (now stored in the variable SERIAL_NUMBER) Determine the USB port used for the phone ... The USB port for the phone with the s/n "M6AIB760D0939LX" is "/dev/bus/usb/003/064" (ASUSTek Computer, Inc. Zenfone GO (ZB500KL) (Debug, MTP mode)) The phone is available via adb - no reset of the USB port necessary Using the phone with the serial number found in the environment variable SERIAL_NUMBER: "M6AIB760D0939LX" Using the TWRP image defined in the environment variable TWRP_IMAGE: "/data/backup/ASUS_ZENFONE8/twrp/current_twrp.img" Using the options "-d -s M6AIB760D0939LX " for the adb commands Using the options " -s M6AIB760D0939LX " for the fastboot commands Checking the script prerequisites ... The status of the phone is 3 - a reset of the USB port is not necessary or usefull The phone is currently booted into the Android OS (PHONE_STATUS=3) The running OS on the phone is "StatixXOS" (Android ) The patch level of the installed OS is "2024-07-05" The build type of the installed OS is "user" The boot slot is _a. The returncode is 3 RC is 3 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

The script boot_phone_from_twrp.sh can also be used as include file using

. ./boot_phone_from_twrp.sh

Using this method these functions are available in the script that includes boot_phone_from_twrp.sh:

check_prereqs_for_boot_phone_from_twrp
init_global_vars_for_boot_phone_from_twrp
set_serial_number
retrieve_phone_status
print_phone_status
convert_phone_status_into_status_string
boot_phone_from_the_TWRP_image
reboot_phone
wait_for_phone_to_be_in_the_bootloader
wait_for_phone_with_a_working_adb_connection
wait_for_the_adb_daemon
wait_until_an_android_service_is_ready
wait_until_data_is_mounted
check_access_via_adb
decrypt_data
umount_data
format_data
wipe_cache
wipe_data
wipe_dalvik
format_metadata
upload_file_to_the_phone
download_file_from_the_phone
copy_file_on_the_phone
start_adb_daemon
kill_adb_daemon
restart_adb_daemon
get_usb_device_for_the_phone
reset_usb_port
reset_the_usb_port_for_the_phone
reset_the_usb_port_for_the_phone_if_necessary
online_adb_connection
print_global_variables
list_running_adb_daemons
enable_safe_mode
boot_phone_from_twrp_die
check_android_boot_image
get_twrp_image_for_the_installed_OS
select_twrp_image_for_install_image

Please check the comments at the start of the script for details about using the script in this way; the command

grep "#S#" boot_phone_from_twrp.sh | cut -c4-

prints the hints for using the script as include file.

Details for using the script as include file

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ grep "#S#" boot_phone_from_twrp.sh | cut -c4- --------------------------------------------------------------------- To use the script as include file add the statement . ./boot_phone_from_twrp.sh to your file Set the variable EXECUTE_BOOT_PHONE_FROM_TWRP_INIT to 1 to disable the initialization code in this script . If used as include file the script - defines these constants if not already defined: __TRUE __FALSE - defines these functions if not already defined: LogMsg LogError LogInfo LogWarning isNumber wait_some_seconds - defines these functions (overwriting existing functions with the same name): check_prereqs_for_boot_phone_from_twrp init_global_vars_for_boot_phone_from_twrp set_serial_number retrieve_phone_status print_phone_status convert_phone_status_into_status_string boot_phone_from_the_TWRP_image reboot_phone wait_for_phone_to_be_in_the_bootloader wait_for_phone_with_a_working_adb_connection wait_for_the_adb_daemon wait_until_an_android_service_is_ready wait_until_data_is_mounted check_access_via_adb decrypt_data umount_data format_data wipe_cache wipe_data wipe_dalvik format_metadata upload_file_to_the_phone download_file_from_the_phone copy_file_on_the_phone start_adb_daemon kill_adb_daemon restart_adb_daemon get_usb_device_for_the_phone reset_usb_port reset_the_usb_port_for_the_phone reset_the_usb_port_for_the_phone_if_necessary online_adb_connection print_global_variables list_running_adb_daemons enable_safe_mode boot_phone_from_twrp_die check_android_boot_image get_twrp_image_for_the_installed_OS select_twrp_image_for_install_image The script using this script as an include file can optional define these variables before including this script (see the source code below for the meaning of these variables): EXECUTE_BOOT_PHONE_FROM_TWRP_INIT SUDO_PREFIX CUR_USER FORCE_BOOT_INTO_TWRP_IMAGE FORCE_REBOOT_INTO_TWRP_IMAGE TWRP_IMAGE SERIAL_NUMBER ADB_DAEMON_WAIT_TIME FASTBOOT_WAIT_TIME ADB_BOOT_WAIT_TIME DECRYPT_DATA_WAIT_TIME ADB_SERVICE_WAIT_TIME FASTBOOT_TIMEOUT FASTBOOT_OPTIONS ADB_OPTIONS ADB_PORT CHECK_ONLY PRINT_STATUS RESET_THE_USB_PORT USER_PASSWORD ADB FASTBOOT TIMEOUT USBRESEST The global variables listed above are defined in any case after including this script. The variable USER_PASSWORD is only used if the phone is protected by a pin or something similar The global variables listed below are set by the functions in this file: Variable Name Contents set by the function ---------------------------------------------------------------------------------------------------------------------------------------- TEMP_TWRP_IMAGE_TO_USE TWRP image that should be used for the installed OS get_twrp_image_for_the_installed_OS, select_twrp_image_for_install_image CURRENT_INSTALLED_ROM Installed ROM on the phone get_twrp_image_for_the_installed_OS CURRENT_INSTALLED_ROM_VERSION version of the installed ROM get_twrp_image_for_the_installed_OS PHONE_STATUS current status of the phone retrieve_phone_status PHONE_STATUS_OUTPUT the last error message regarding the status of the phone retrieve_phone_status PROP_RO_TWRP_BOOT the value of the property.ro.twrp.boot retrieve_phone_status PROP_RO_BOOTMODE the value of the property ro.bootmode retrieve_phone_status PROP_RO_BUILD_DESCRIPTION the value of the property property ro.build.description retrieve_phone_status PROP_RO_PRODUCT_BUILD_VERSION_RELEASE the value of the property ro.product.build.version.release retrieve_phone_status BOOT_SLOT the current boot slot : either _a or _b or empty retrieve_phone_status INACTIVE_SLOT the inactive slot : either _a or _b or empty retrieve_phone_status RUNNING_IN_AN_ADB_SESSIION is ${__TRUE} if the script is running in an adb session retrieve_phone_status ROM_NAME name of the OS running on the phone retrieve_phone_status PHONE_IN_EDL_MODE lsusb entry for phones in EDL mode retrieve_phone_status PHONE_USB_DEVICE path to the USB port used for the phone retrieve_phone_status PHONE_BOOT_ERROR contains the error code if booting the phone fails boot_phone_from_the_TWRP_image OS_BUILD_TYPE contains the build of the installed OS (eng, userdebug, or user) retrieve_phone_status OS_PATCH_LEVEL contains the patch level of the installed OS retrieve_phone_status PROP_RO_SECURE the value of the property ro.secure retrieve_phone_status PROP_RO_DEBUGGABLE the value of the property ro.debuggable retrieve_phone_status PROP_RO_MODVERSION the value of the property ro.modversion retrieve_phone_status ------------------------------------------------------------------------------------------------------------------------------ The ROMs known by the function "select_twrp_image_for_install_image" are read from the variable TWRP_IMAGES_FOR_IMAGE_FILES The format of the entries in the variable TWRP_IMAGES_FOR_IMAGE_FILES is: each line must contain these fields (the field separator is a colon ":"; lines starting with a hash "#" are ignored) : regex for the image file : TWRP image to use : Name of the ROM : description The fields for the name of the ROM and the description are optional The regex can only contain the joker characters "?" and "*" ------------------------------------------------------------------------------------------------------------------------------ The ROMs known by the function "get_twrp_image_for_the_installed_OS" is read from the variable TWRP_IMAGES_FOR_THE_RUNNING_OS The format of the entries in the variable TWRP_IMAGES_FOR_THE_RUNNING_OS is: each line must contain these fields (the field separator is a colon ":"; lines starting with a hash "#" are ignored) : property : value : TWRP image to use : Name of the ROM : description Use "*" for the field "value" if the property must be defined but the value is meaningless The fields for the name of the ROM and the description are optional See below for examples. Note: The function "get_twrp_image_for_the_installed_OS" is used to get the TWRP image to use if the script is called without a parameter for the TWRP image to use. If the function does not find a TWRP image for the running OS, the TWRP image defined in the variable TWRP_IMAGE is used. The default value for the variable TWRP_IMAGE is the value of the variable DEFAULT_TWRP_IMAGE hardcoded in the script ------------------------------------------------------------------------------------------------------------------------------ The ROMs known by the function "retrieve_phone_status" script are: ASUS Android OmniROM LineageOS StatiXOS LMODroid /e/ ------------------------------------------------------------------------------------------------------------------------------ The status of the phone is stored in the global variable PHONE_STATUS: 1 - the phone is already booted from the TWRP image 2 - the phone is booted from TWRP installed in the boot or recovery partition 3 - the phone is booted into the Android OS 4 - the phone is booted into bootloader 5 - the phone is booted into the fastbootd 6 - the phone is booted into the safe mode of the Android OS 7 - the phone is booted into the a non-TWRP recovery installed in the boot or recovery partition 8 - the phone is booted into sideload mode 9 - the phone is booted into a recovery without working adb shell 10 - error retrieving the status of the attached phone (or no phone connected) 11 - the phone is attached but without permissions to access via fastboot or adb 12 - the phone seems to be in EDL mode [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Note:

After including the script you should use the command

${ADB} ${ADB_OPTIONS}

to run the adb binary and the command

${FASTBOOT} ${FASTBOOT_OPTIONS}

to run the fastboot binary.

Examples:

To boot the phone from the TWRP image use this code:

. ./boot_phone_from_twrp.sh retrieve_phone_status boot_phone_from_the_TWRP_image

To boot the phone into the Android OS use this code:

. ./boot_phone_from_twrp.sh retrieve_phone_status reboot_phone

Simple Test script

[xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ cat ./test.sh echo "*** Loading the functions to boot the phone ..." . ./boot_phone_from_twrp.sh echo "" echo "*** Checking the status of the phone" retrieve_phone_status print_phone_status echo "" echo "*** Booting the phone into the TWRP image ..!" boot_phone_from_the_TWRP_image echo "" echo "*** Press return to boot the phone into the Android OS ..." read USER_INPUT reboot_phone [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ ./test.sh *** Loading the functions to boot the phone ... The script is running as user "xtrnaw7" -- will use "sudo" for the fastboot commands ... Will use the attached phone with the serial number "M6AIB760D0939LX" Using the default TWRP image: "/data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D.img" Using the options " -s M6AIB760D0939LX " for the adb commands Using the options " -s M6AIB760D0939LX " for the fastboot commands Checking the script prerequisites ... Will wait up to 60 second(s) after booting the phone into the bootloader Will wait up to 60 second(s) after booting the phone from the TWRP image Will wait up to 10 second(s) until the adb daemon is ready to use *** Checking the status of the phone The phone is currently booted from the Android OS *** Booting the phone into the TWRP image ..! The phone is booted into the Android OS Booting the phone into the bootloader now ... Waiting up to 60 seconds for the boot into the fastboot mode ... The phone is booted into the bootloader Booting the phone from the TWRP image "/data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D.img" now ... Sending 'boot.img' (98304 KB) OKAY [ 2.761s] Booting OKAY [ 10.361s] Finished. Total time: 13.167s Waiting up to 60 seconds for the boot of the phone .... Waiting up to 10 seconds for the adb daemon to get ready .. ... the adb daemon is ready after second(s) The phone is booted into TWRP: "twrp_I006D" version "3.7.0_12-0" *** Press return to boot the phone into the Android OS ... Booting the Android OS using the command "adb reboot " now ... Waiting up to 60 seconds for the boot of the phone ..... Waiting up to 10 seconds for the adb daemon to get ready .......... [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$

The script boot_phone_from_twrp.sh is used as include script in the script init_magisk_db.sh - see also this post How to enable root access using Magisk in a script.
The script is also used in the script install_os_via_twrp.sh see this post How to install an OS image using the twrp command in a script
The script is available on my homepage:

https://bnsmb.de/files/public/Android//boot_phone_from_twrp.sh

Changelog for the script boot_phone_from_twrp.sh

# # History # 05.11.2022 v1.0.0.0 /bs #VERSION # inital release # # 01.12.2022 v2.0.0.0 /bs #VERSION # script rewritten using functions # added the parameter --serial=#sn # # 07.12.2022 v2.1.0.0 /bs #VERSION # added a check for the Android OS safe mode # missing shift command in the code to process the parameter for the serial number -- fixed # # 17.12.2022 v2.2.0.0 /bs #VERSION # added support for the LineageOS recovery # use the command timeout for "su -" commands on the phone # # 01.01.2023 v2.3.0.0 /bs #VERSION # added support for a parameter for the TWRP image file to use for the function "boot_phone_from_the_TWRP_image" # added support for the parameter "force" to the function "reboot_phone" # added support for the phone mode "sideload" # function "reboot_phone" rewritten from sratch # the script now first checks for phones connected via adb and then for phones connected via fastboot # the check for safe mode now also works without root access # the script now ends with return code 110 instead of 10 if executed with the parameter "-h" # the script now also prints the current boot slot if executed with the parameter "status" # # 04.01.2023 v2.3.1.0 /bs #VERSION # the function reboot_phone now uses fastboot to check if the boot into the bootloader was successfull # function reboot_phone: replaced "adb wait-for..." commands with other commands ; these "adb wait-for.." commands are not reliable # added the returncode "9 - the phone is booted into a recovery without working adb shell" to the function to check the phone status # # 08.01.2023 v2.3.1.1 /bs #VERSION # the script now also supports the parameter with leading "--" # # 14.01.2023 v2.3.1.2 /bs #VERSION # increased the timeout to wait for booting into the TWRP image from 60 to 120 seconds # increased the timeout to wait for the adb to be ready after booting into TWRP from 10 to 30 seconds # # 16.01.2023 v2.3.2.0 /bs #VERSION # added a parameter for the time to wait for booting the phone to the function reboot_phone # # 18.01.2023 v2.3.2.1 /bs #VERSION # fixed a minor issue in the messages written by the function wait_for_the_adb_daemon # # 20.01.2023 v2.3.2.2 /bs #VERSION # the script ignored the values of the envionment variables ADB_OPTIONS and FASTBOOT_OPTIONS (these variables were deleted in the function set_serial_number ) -- fixed # # 28.01.2023 v2.3.2.3 /bs #VERSION # the code to process the parameter s=# was wrong -- fixed # # 23.04.2023 v2.4.0.0 /bs #VERSION # added the function decrypt_data to decrypt an encrypted data partition # added the parameter password=passwd # added the support for the global variable USER_PASSWORD # added the parameter Android, android, recovery, bootloader, sideload, fastboot, and safemode # added the function copy_file_on_the_phone # added the function download_file_from_the_phone # added the function upload_file_to_the_phone # # 01.05.2023 v2.5.0.0 /bs #VERSION # the function reboot_phone did not work if called without a parameter for the boot target -- fixed # the function decrypt_data now uses the Android command "input" to "decrypt" the data parition if booted into the Android OS # added the function wipe_cache # added the function wipe_dalvik # added the function umount_data # added the function format_data # added the function wipe_data # added the function format_metadata # added the function kill_adb_daemon # # 10.05.2023 v2.5.1.0 /bs #VERSION # fixed syntax errors in the function wait_until_data_is_mounted # the function for booting the phone now returns an error if decrypting the /data partition failed # the function for booting the phone now returns an error if decrypting the /data partition failed # # 02.07.2023 v2.5.2.0 /bs #VERSION # the variable DEFAULT_PASSWORD can be used to define the default password for file encyrption in Android # the function decrypt_data now uses the default password decrypting files if no user password was set # # 22.11.2023 v2.5.3.0 /bs #VERSION # the default password for decyrpting the /datas partition was never used due to a wrong if clause in the function decrypt_data -- fixed # # 27.12.2023 v2.5.3.1 /bs #VERSION # egrep replaced with "grep -E" # # 27.04.2024 v2.5.3.2 /bs #VERSION # increase the timeout for waiting for decryption in the function decrypt_data from 15 to 150 seconds # # 28.04.2024 v2.5.3.3 /bs #VERSION # the function wait_until_data_is_mounted always returned ${__TRUE} because of a typo -- fixed # # 29.04.2024 v2.5.4.0 /bs #VERSION # the parameter for the TWRP image file did not work if the environment variable TWRP_IMAGE was already set before starting the script -- fixed # improved the message about the source for the TWRP image used # # 30.04.2024 v2.6.0.0 /bs #VERSION # the script does not try to decrypt a not initialized /data partition anymore # added the function get_twrp_image_for_the_installed_OS and the global variables TEMP_TWRP_IMAGE_TO_USE and CURRENT_INSTALLED_ROM # added the function online_adb_connection # added more comments with details for the usage of this file # added the alias check for the parameter checkonly # the function format_data did not work as expected due to a invalid twrp command used - fixed # # 03.05.2024 v2.6.1.0 /bs #VERSION # the script does not check the phone status when starting anymore if the variable CHECK_THE_PHONE_STATUS is set to ${__FALSE} # (the variable CHECK_THE_PHONE_STATUS is used prepare_phone.include) # added the function restart_adb_daemon # the functions start_adb_daemon and restart_adb_daemon now support the parameter "root" to start/restart the daemon via sudo # changed the grep command to check if the adb daemon is running in all functions to ' grep " adb " ' # # 05.05.2024 v2.6.2.0 /bs #VERSION # added support for /e/ # the function reboot_phone ended with a syntax error for unknown phone states -- fixed # # 05.05.2024 v2.6.3.0 /bs #VERSION # the function get_twrp_image_for_the_installed_OS now uses the definitions from the variable TWRP_IMAGES_FOR_THE_RUNNING_OS # # 07.05.2024 v2.6.4.0 /bs #VERSION # added the functions select_twrp_image_for_install_image, check_android_boot_image, LogInfo, and LogWarning # added the funcdtion convert_phone_status_into_status_string # # 08.05.2024 v2.7.0.0 /bs #VERSION # added the global variables ROM_NAME, PROP_RO_BUILD_DESCRIPTION, PROP_RO_PRODUCT_BUILD_VERSION_RELEASE # retrieve_phone_status re-written, PHONE_STATUS=7 is now for a phone booted into the non-TWRP recovery on the phone # the global variable ROM_NAME contains the name of the running OS on the phone # # 15.05.2024 v2.8.0.0 /bs #VERSION # added code to check for phones in EDL mode # added the PHONE_STATUS 12 for phones that seems to be in EDL mode # # 22.05.2024 v3.0.0.0 /bs #VERSION # added code to retrieve the USB port used for the phone # added the parameter usb_reset to force a reset of the USB Device if access to the phone is not working # added the functions reset_the_usb_port_for_the_phone and reset_the_usb_port_for_the_phone_if_necessary # added the global variables USBRESET, RESET_THE_USB_PORT, PHONE_USB_IDENTIFIER, and PHONE_USB_DEVICE # renamed the variable CHECKONLY to CHECK_ONLY # PRINT_STATUS is now a global variable that can be set before starting the script # LogInfo was wrong implemented -- fixed # fixed some errors in the function to detect the phone # added more debug messages -- set the variable VERBOSE to any value before calling the script to print debug messages # the script prints now no message anymore if the parameter "check" is used # added the environment variable EXECUTE_BOOT_PHONE_FROM_TWRP_INIT # the script now boots the TWRP image that is necessary for the running OS if the access via adb is working and the running OS is known by the script # the script now reset the usb port while waiting for an adb connection if necessary and the parameter "reset_usb" is used # there was a typo so that the script did not detect that the phone was already booted from the recovery -- fixed # fixed various other minor errors # function wait_for_phone_with_a_working_adb_connection rewritten from sratch # added the function wait_until_an_android_service_is_ready # the script now also detects the ASUS Android Beta versions # # 25.05.2024 v3.0.1.0 /bs #VERSION # added support for /e/ 2.0 # removed the use of the not defined function LogErrorMsg # added the environment variable ADB_PORT for the port used by the running adb daemon # the function restart_adb_daemon now restarts the adb daemon via systemd if it was started using systemd # # 26.06.2024 v3.0.2.0 /bs #VERSION # added the global variable INACTIVE_SLOT # the function get_twrp_image_for_the_installed_OS now also works if the phone is boot into TWRP or a recovery with enable adb access # # 03.07.2024 v3.0.3.0 /bs #VERSION # added the global variables PROP_RO_MODVERSION, OS_BUILD_TYPE, OS_PATCH_LEVEL, PROP_RO_SECURE, and PROP_RO_DEBUGGABLE,

History of this entry

17.12.2022

Updated the post with the infos for the script version 2.2.0.0

18.12.2022

Added some details for using the script as include file
Added a link to another script using this script

14.07.2024

Updated the script details to match the current script version

How to restore the boot partition to remove Magisk

URL: https://xdaforums.com/t/how-to-restore-the-boot-partition-to-remove-magisk.4515759/

How to restore the boot partition to remove Magisk

Magisk creates a backup of the existing boot partition before rewriting the new boot partition.

This one-liner can be used as user root on the phone to manually remove Magisk from the boot partition by restoring the boot partition from the backup created by Magisk:

gzip -cd /data/magisk_backup_$( grep SHA1 $( magisk --path )/.magisk/config | cut -f2 -d "=" )/boot.img.gz | dd of=/dev/block/by-name/boot$( getprop ro.boot.slot_suffix )

Note:

In production environments a script with some more error checking should be used for this task, of course.

How to compile the OmniROM 13 for the ASUS Zenfone 8

URL: https://xdaforums.com/t/how-to-compile-the-omnirom-13-for-the-android-zenfone-8.4518733/

How to compile the OmniROM 13 for the ASUS Zenfone 8

Compiling the CustomROM OmniROM 13 from within Linux is again a very simple and straight forward process -- the developer for the OmniROM did once again a really amazing good job setting up the build environment for the OmniROM 13.

Notes:

As of 22.01.2023 there is an additional repository required for compiling OmniROM 13 -- see below for details. I also added some more details at the end of this post.

This HowTo is about how to compile the OmniROM 13 for the ASUS Zenfone 8 -- see How to compile the OmniROM for the ASUS Zenfone 8 for how to compile the OmniROM 12 for the ASUS Zenfone 8.

Most (but not all) of the necessarry steps to sync the repositories are described in the section "Getting Started" here: https://github.com/omnirom/android.

Prerequisites

Some of the repositories for OmniROM 13 are on https://gitlab.com and the access to these repositories for the repository sync is done via ssh. Therefor to sync these repositories a user account on gitlab.com with configured public ssh key is required. The user account on gitlab.com is free of charge and the public ssh key can be uploaded into the user profile on gitlab.com (see here: https://docs.gitlab.com/ee/user/ssh.html)

The repositories on http://github.com are currently accessed via https but that might change in the future. Therefor it's recommended to create a user on github.com and add the ssh public key to the profile of that user also.

Note:

To check how the access to the repositories is configured use this command:

xtrnaw7@t15g /data/develop/android/OmniROM_13.0/.repo]$ grep fetch $( find . -name "*xml" ) ./local_manifests/local_manifest.xml: fetch="https://github.com/DonkeyCoyote" /> ./manifests/omni-aosp.xml: fetch="https://github.com/" /> ./manifests/omni-aosp.xml: fetch="ssh://git@gitlab.com/omnirom" /> ./manifests/omni-aosp.xml: fetch="https://github.com/omnirom" ./manifests/omni-caf.xml: fetch="https://github.com/" /> ./manifests/omni-caf.xml: fetch="https://github.com/omnirom" ./manifests/omni-caf.xml: fetch="https://git.codelinaro.org/clo/la" ./manifests/omni-private.xml: fetch="ssh://git@github.com/omnirom" ./manifests/default.xml: fetch="https://android.googlesource.com" [xtrnaw7@t15g /data/develop/android/OmniROM_13.0/.repo]$
The tool repo is required to maintain the repositories for the OmniROM 13; you can download the tool "repo" from https://gerrit.googlesource.com/git-repo

It's not necessary to install additional tools for creating an OmniROM image -- all necessary tools to create the image are in the repositories for the OmniROM 13.

Creating the local copies of the repositories for the OmniROM 13

To create a local copy of the repositories for OmniROM 13 these steps are required:

First create a new directory for the repositories and change the working directory to the new directory, e.g.:

mkdir /data/develop/android/OmniROM_13.0 cd /data/develop/android/OmniROM_13.0

Init the local repositories using the command:

repo init -u https://github.com/omnirom/android.git -b
        android-13.0

Note that the command repo init only creates a hidden directory called .repo.

Next do the initial sync of the repositories using the command:



        repo sync

Be aware that the repositories need about 160 GB disk space so this will take a while.

The initial repository sync will download all repositories necessary to create the OmniROM image except the hardware dependent repositories (these repos are different for the various phone models).

Therefor you must add the repositories for the vendor and device tree for your phone to the manifests in the local repositories after the download finished.

This should be done in the file .repo/local_manifests/local_manifest.xml (if the directory .repo/local_manifests does not exist create it).

For the ASUS Zenfone 8 the file should look like this:

[xtrnaw7@t15g /data/develop/android/OmniROM_13.0]$ cat
        .repo/local_manifests/local_manifest.xml 

        <?xml version="1.0" encoding="UTF-8"?>

        

        <manifest>

        <remote name="DonkeyCoyote"

        fetch="https://github.com/DonkeyCoyote" />

        

        <project path="device/asus/zenfone8"
        name="android_device_asus_zenfone8" remote="omnirom"
        revision="android-13.0" />

        <project path="vendor/asus" name="proprietary_vendor_asus"
        remote="DonkeyCoyote" revision="android-13.0" />

        <project path="vendor/microg"
        name="android_prebuilts_prebuiltapks" remote="omnirom"
        revision="android-13.0" />

        <project path="vendor/images/asus"
        name="android_vendor_images_asus" remote="gitlab"
        revision="android-13.0" />

        </manifest>

        [xtrnaw7@t15g /data/develop/android/OmniROM_13.0]$

Note:

The repository android_prebuilts_prebuiltapks is only necessary for OmniROM images that should include the MicroG packages.

Update 22.01.2023

The repository android_vendor_images_asus is now also required; currently this repository only contains the ASUS Galery.
I also added some more details at the end of this post.

After updating the file local_manifest.xml do another

repo sync

to sync the added repositories. This time the download will be faster because only the two (or three) additional repos will be downloaded.

Now you can start the build with these commands:

# optional:

        #

        # to create an OmniROM image with MicroG packages set the
        environment variable ROM_BUILDTYPE to MICROG before starting the
        build

        #

        # to create an OmniROM image with GAPPS packages set the
        environment variable ROM_BUILDTYPE to GAPPS before starting the
        build

        #(that will only work if the repositories for the GAPPS are
        configured, of course)

        #

        export ROM_BUILDTYPE=MICROG

        

        . build/envsetup.sh

        

        brunch omni_zenfone8-user

in the directory with the downloaded repositories (use the command "breakast" instead of "brunch" to see other possible targets)

Note that the first time the compile process will take 4 hours or more. And keep in mind that the compile process will need additional 140 GB for the temporary output files.

If everything is fine there will be a ZIP file with your own CustomROM image in the directory ./out/target/product/zenfone8/ when the compile script is done, e.g. for images with the MicroG packages:

[xtrnaw7@t15g /data/develop/android/OmniROM_13.0]$ ls -l
        out/target/product/zenfone8/omni-13-2022*-zenfone8-*.zip

        -rw-rw-r--. 3 xtrnaw7 xtrnaw7 1258920226 Nov 14 11:55
out/target/product/zenfone8/omni-13-202211141148-zenfone8-MICROG.zip

        [xtrnaw7@t15g /data/develop/android/OmniROM_13.0]$

The image can now be installed on the ASUS Zenfone 8 using the LineageOS Recovery or TWRP.

Notes

Be aware that each new ROM image created with brunch will overwrite the existing ROM images (even if the name of the image is different), e.g.

[xtrnaw7@t15g /data/develop/android/OmniROM_13.0]$ ls -li out/target/product/zenfone8/omni-13-2022*-zenfone8-*.zip 77928215 -rw-rw-r--. 3 xtrnaw7 xtrnaw7 1258920226 Nov 14 11:55 out/target/product/zenfone8/omni-13-202211131612-zenfone8-MICROG.zip 77928215 -rw-rw-r--. 3 xtrnaw7 xtrnaw7 1258920226 Nov 14 11:55 out/target/product/zenfone8/omni-13-202211141148-zenfone8-MICROG.zip [xtrnaw7@t15g /data/develop/android/OmniROM_13.0]$ [xtrnaw7@t15g /data/develop/android/OmniROM_13.0]$ cksum out/target/product/zenfone8/omni-13-2022*-zenfone8-*.zip 2429251380 1258920226 out/target/product/zenfone8/omni-13-202211131612-zenfone8-MICROG.zip 2429251380 1258920226 out/target/product/zenfone8/omni-13-202211141148-zenfone8-MICROG.zip [xtrnaw7@t15g /data/develop/android/OmniROM_13.0]$

Therefor you should copy the zip file with the new image to a new directory before building new image

For OS images that should be used on the phone for daily use I strongly recommend to create your own certificates for the OS.

In the OmniROM development directory tree the neccessary files, platform.pk8 and platform.x509.pem, are in the directory

./build/make/target/product/security

in your build tree for the CustomROM, e.g.

[xtrnaw7@t15g /data/develop/android/OmniROM]$ ls -l
          ./build/make/target/product/security/platform.*
        -rw-------. 1 xtrnaw7 xtrnaw7 1219 Jun 25 09:39
        ./build/make/target/product/security/platform.pk8 -rw-rw-r--. 1
        xtrnaw7 xtrnaw7 1460 Jun 25 09:39
        ./build/make/target/product/security/platform.x509.pem
        [xtrnaw7@t15g /data/develop/android/OmniROM]$

Replace these files with your own certificate files and recreate the image using the command brunch (see above).

See here https://source.android.com/devices/tech/ota/sign_builds for how to create your own certificate.

How to manually create the SSL keys

To manually create the keys use these commands:

Android uses 2048-bit RSA keys with public exponent 3. You can generate certificate/private key pairs using the openssl tool from openssl.org:

# generate RSA key #[xtrnaw7@t15g /data/develop/android/test1]$ openssl genrsa -3 -out temp.pem 2048 [xtrnaw7@t15g /data/develop/android/test1]$ ls -ltr total 4 -rw-------. 1 xtrnaw7 xtrnaw7 1704 Nov 14 13:24 temp.pem # create a certificate with the public part of the key # [xtrnaw7@t15g /data/develop/android/test1]$ openssl req -new -x509 -key temp.pem -out releasekey.x509.pem -days 10000 -subj '/C=DE/ST=Hessen/L=Frankfurt am Main/O=Android/OU=Android/CN=Android/emailAddress=bnsmb01@gmail.com' [xtrnaw7@t15g /data/develop/android/test1]$ [xtrnaw7@t15g /data/develop/android/test1]$ ls -ltr total 8 -rw-------. 1 xtrnaw7 xtrnaw7 1704 Nov 14 13:24 temp.pem -rw-rw-r--. 1 xtrnaw7 xtrnaw7 1456 Nov 14 13:25 releasekey.x509.pem [xtrnaw7@t15g /data/develop/android/test1]$ # create a PKCS#8-formatted version of the private key # [xtrnaw7@t15g /data/develop/android/test1]$ openssl pkcs8 -in temp.pem -topk8 -outform DER -out releasekey.pk8 -nocrypt [xtrnaw7@t15g /data/develop/android/test1]$ [xtrnaw7@t15g /data/develop/android/test1]$ ls -ltr total 12 -rw-------. 1 xtrnaw7 xtrnaw7 1704 Nov 14 13:24 temp.pem -rw-rw-r--. 1 xtrnaw7 xtrnaw7 1456 Nov 14 13:25 releasekey.x509.pem -rw-------. 1 xtrnaw7 xtrnaw7 1218 Nov 14 13:25 releasekey.pk8 [xtrnaw7@t15g /data/develop/android/test1]$ # securely delete the temp.pem file #[xtrnaw7@t15g /data/develop/android/test1]$ shred --remove temp.pem [xtrnaw7@t15g /data/develop/android/test1]$ [xtrnaw7@t15g /data/develop/android/test1]$ ls -ltr total 8 -rw-rw-r--. 1 xtrnaw7 xtrnaw7 1456 Nov 14 13:25 releasekey.x509.pem -rw-------. 1 xtrnaw7 xtrnaw7 1218 Nov 14 13:25 releasekey.pk8 [xtrnaw7@t15g /data/develop/android/test1]$
From the documentation:

The openssl pkcs8 command given above creates a .pk8 file with no password, suitable for use with the build system. To create a .pk8 secured with a password (which you should do for all actual release keys), replace the -nocrypt argument with -passout stdin; then openssl will encrypt the private key with a password read from standard input. No prompt is printed, so if stdin is the terminal the program will appear to hang when it's really just waiting for you to enter a password. Other values can be used for the-passout argument to read the password from other locations; for details, see the openssl documentation.

The temp.pem intermediate file contains the private key without any kind of password protection, so dispose of it thoughtfully when generating release keys. In particular, the GNUshred utility may not be effective on network or journaled filesystems. You can use a working directory located in a RAM disk (such as a tmpfs partition) when generating keys to ensure the intermediates are not inadvertently exposed.

Updating the local repositories [Update 22.01.2023]

To update the local repositories change the working directory to the directory with the repositories for OmniROM 13 and execute

repo sync

After the synchronization is complete, create a new operating system image using the commands described above.

It is recommended to create a new image at least once a month after the new security patches for Android are added to the repositories for OmniROM.

To view the date of the security patch in the source files for the OmniROM do:

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_13.0 ] $ grep "^[[:space:]]*PLATFORM_SECURITY_PATCH[[:space:]]*:=" ./build/make/core/version_defaults.mk PLATFORM_SECURITY_PATCH := 2023-01-05[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_13.0 ] $

Moving the repositories to another directory [Update 22.01.2023]

In case you want to move the repositories to another directory or rename the directory with the repositories:

Be aware that the make scripts create some absolute links in the output directory; to list these absolute links use while in the directory with the repositories:

find . -type l -ls | grep "$PWD"

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_13.0 ] $ find . -type l -ls | grep "$PWD" 79960553 4 lrwxrwxrwx 1 xtrnaw7 xtrnaw7 93 Dec 18 18:49 ./out/soong/.intermediates/prebuilts/build-tools/prebuilt_py3-launcher-autorun/linux_glibc_x86_64/py3-launcher-autorun -> /data/develop/android/OmniROM_13.0/prebuilts/build-tools/linux-x86/bin/py3-launcher-autorun64 79960558 4 lrwxrwxrwx 1 xtrnaw7 xtrnaw7 73 Dec 18 18:46 ./out/soong/.intermediates/prebuilts/build-tools/m4/linux_glibc_x86_64/m4 -> /data/develop/android/OmniROM_13.0/prebuilts/build-tools/linux-x86/bin/m4 78782027 0 lrwxrwxrwx 1 xtrnaw7 xtrnaw7 53 Jan 22 09:07 ./out/target/product/zenfone8/obj/DTBO_OBJ/source -> /data/develop/android/OmniROM_13.0/kernel/asus/sm8350 78931423 0 lrwxrwxrwx 1 xtrnaw7 xtrnaw7 53 Jan 22 09:07 ./out/target/product/zenfone8/obj/DTB_OBJ/source -> /data/develop/android/OmniROM_13.0/kernel/asus/sm8350 78792528 0 lrwxrwxrwx 1 xtrnaw7 xtrnaw7 53 Jan 15 11:56 ./out/target/product/zenfone8/obj/PACKAGING/kernel_modules_intermediates/lib/modules/5.4.147-Omni-qgki-perf-g3e8f0551c618/source -> /data/develop/android/OmniROM_13.0/kernel/asus/sm8350 78792529 4 lrwxrwxrwx 1 xtrnaw7 xtrnaw7 77 Jan 15 11:56 ./out/target/product/zenfone8/obj/PACKAGING/kernel_modules_intermediates/lib/modules/5.4.147-Omni-qgki-perf-g3e8f0551c618/build -> /data/develop/android/OmniROM_13.0/out/target/product/zenfone8/obj/KERNEL_OBJ 84806159 0 lrwxrwxrwx 1 xtrnaw7 xtrnaw7 53 Jan 22 09:13 ./out/target/product/zenfone8/obj/PACKAGING/kernel_modules_intermediates/lib/modules/5.4.147-Omni-qgki-perf-g875131b91949/source -> /data/develop/android/OmniROM_13.0/kernel/asus/sm8350 84806160 4 lrwxrwxrwx 1 xtrnaw7 xtrnaw7 77 Jan 21 11:25 ./out/target/product/zenfone8/obj/PACKAGING/kernel_modules_intermediates/lib/modules/5.4.147-Omni-qgki-perf-g875131b91949/build -> /data/develop/android/OmniROM_13.0/out/target/product/zenfone8/obj/KERNEL_OBJ 78788578 0 lrwxrwxrwx 1 xtrnaw7 xtrnaw7 53 Jan 22 09:10 ./out/target/product/zenfone8/obj/KERNEL_OBJ/source -> /data/develop/android/OmniROM_13.0/kernel/asus/sm8350 [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_13.0 ] $

Before starting another build after moving the directories with the repositories you must correct these links.

Trouble Shooting

Update 08.02.2024

see Some hints for repo sync troubleshooting for how to fix common errors executing repo sync.

see Some hints for fixing errros building an Custom ROM Android image for how to fix some common errors while creating the image

How to change files in Android XML Binary format

URL: https://xdaforums.com/t/how-to-change-files-in-android-xml-binary-format.4521513/

How to change global settings via script in Android

In the OmniROM (tested in OmniROM 12 and OmniROM 13) - and I assume in other Android distributions also - global settings are stored the file

/data/system/users/0/settings_global.xml

The file settings_global.xml is in the Android Binary XML format and can not be edited via text editor. To change the entries in that file via script it must be converted to text format and
after the necessary changes are done in the file in human readable format converted back from text format to the Android Binary XML format. This can be done with the Android executables abx2xml and xml2abx:

ASUS_I006D:/ # abx2xml -h java.lang.ArrayIndexOutOfBoundsException: length=2; index=2 usage: abx2xml [-i] input [output] usage: xml2abx [-i] input [output] Converts between human-readable XML and Android Binary XML. When invoked with the '-i' argument, the output of a successful conversion will overwrite the original input file. Input can be '-' to use stdin, and output can be '-' to use stdout. ASUS_I006D:/ #

Note:

See here https://www.cclsolutionsgroup.com/post/android-abx-binary-xml or here https://cs.android.com/android/platform/superproject/+/master:frameworks/base/core/java/com/android/internal/util/BinaryXmlSerializer.java for details about the Android Binary XML file format.

A lot of xml files in Android are in the Android Binary XML format (there may be others on other phones):

ASUS_I006D:/ # find /data/system -name "*.xml" -exec file {} \; 2>/dev/null | grep "Android Binary XML" | wc -l 31 ASUS_I006D:/ #

find /data/system -name "*.xml" -exec file {} \; 2>/dev/null | grep "Android Binary XML"

ASUS_I006D:/ # find /data/system -name "*.xml" -exec file {} \; 2>/dev/null | grep "Android Binary XML"/data/system/users/0/package-restrictions.xml: Android Binary XML v0 /data/system/users/0/registered_services/android.content.SyncAdapter.xml: Android Binary XML v0 /data/system/users/0/registered_services/android.accounts.AccountAuthenticator.xml: Android Binary XML v0 /data/system/users/0/settings_system.xml: Android Binary XML v0 /data/system/users/0/settings_secure.xml: Android Binary XML v0 /data/system/users/0/settings_global.xml: Android Binary XML v0 /data/system/users/0/wallpaper_info.xml: Android Binary XML v0 /data/system/users/0/appwidgets.xml: Android Binary XML v0 /data/system/users/0.xml: Android Binary XML v0 /data/system/users/userlist.xml: Android Binary XML v0 /data/system/appops/history/900000.xml: Android Binary XML v0 /data/system/appops/history/9900000.xml: Android Binary XML v0 /data/system/appops/history/99900000.xml: Android Binary XML v0 /data/system/package-watchdog.xml: Android Binary XML v0 /data/system/display-manager-state.xml: Android Binary XML v0 /data/system/packages.xml: Android Binary XML v0 /data/system/overlays.xml: Android Binary XML v0 /data/system/sensor_privacy.xml: Android Binary XML v0 /data/system/watchlist_settings.xml: Android Binary XML v0 /data/system/cachequota.xml: Android Binary XML v0 /data/system/job/jobs.xml: Android Binary XML v0 /data/system/device_policies.xml: Android Binary XML v0 /data/system/shortcut_service.xml: Android Binary XML v0 /data/system/inputmethod/subtypes.xml: Android Binary XML v0 /data/system/install_sessions.xml: Android Binary XML v0 /data/system/appops.xml: Android Binary XML v0 /data/system/log-files.xml: Android Binary XML v0 /data/system/notification_policy.xml: Android Binary XML v0 /data/system/netpolicy.xml: Android Binary XML v0 /data/system/sensor_privacy_impl.xml: Android Binary XML v0 /data/system/batterystats-daily.xml: Android Binary XML v0 ASUS_I006D:/ #

So to change the settings in the file settings_global.xml these commands are necessary:

Example:

# convert the file into a human readable format # ASUS_I006D:/ # abx2xml /data/system/users/0/settings_global.xml /data/local/tmp/setttings_global.xml.human ASUS_I006D:/ # ASUS_I006D:/ # ls -l /data/local/tmp/setttings_global.xml.human -rw-r--r-- 1 root root 15425 2022-11-20 14:30 /data/local/tmp/setttings_global.xml.human ASUS_I006D:/ # ASUS_I006D:/ # file /data/local/tmp/setttings_global.xml.human /data/local/tmp/setttings_global.xml.human: ASCII text ASUS_I006D:/ # ASUS_I006D:/ # head -5 /data/local/tmp/setttings_global.xml.human<?xml version='1.0' encoding='UTF-8' standalone='yes' ?> <settings version="210"> <setting id="140" name="adb_wifi_enabled" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="86" name="mobile_data1" value="0" package="com.android.phone" defaultValue="0" defaultSysSet="true" /> <setting id="91" name="mobile_data2" value="0" package="com.android.phone" defaultValue="0" defaultSysSet="true" preserve_in_restore="true" /> ASUS_I006D:/ #

Now edit the file /data/local/tmp/setttings_global.xml.human and convert it back to Android Binary XML format

# conver the XML file in human readable format into the Android Binary XML format # ASUS_I006D:/ # xml2abx /data/local/tmp/setttings_global.xml.human /data/system/users/0/settings_global.xmlASUS_I006D:/ # ASUS_I006D:/ # ls -l /data/system/users/0/settings_global.xml -rw------- 1 system system 10697 2022-11-20 15:34 /data/system/users/0/settings_global.xml ASUS_I006D:/ #

To activate the new settings just reboot the phone.

Notes:

The global settings in the file settings_global.xml in the OmniROM 13 are (there may be others on other phones):

cat /data/local/tmp/setttings_global.xml.human

ASUS_I006D:/ # cat /data/local/tmp/setttings_global.xml.human<?xml version='1.0' encoding='UTF-8' standalone='yes' ?> <settings version="210"> <setting id="140" name="adb_wifi_enabled" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="86" name="mobile_data1" value="0" package="com.android.phone" defaultValue="0" defaultSysSet="true" /> <setting id="91" name="mobile_data2" value="0" package="com.android.phone" defaultValue="0" defaultSysSet="true" preserve_in_restore="true" /> <setting id="41" name="low_battery_sound_timeout" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="33" name="car_undock_sound" value="/product/media/audio/ui/Undock.ogg" package="android" defaultValue="/product/media/audio/ui/Undock.ogg" defaultSysSet="true" /> <setting id="131" name="enable_freeform_support" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" /> <setting id="128" name="show_notification_channel_warnings" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" /> <setting id="103" name="time_remaining_estimate_based_on_usage" value="0" package="com.android.systemui" defaultValue="0" defaultSysSet="true" preserve_in_restore="true" /> <setting id="142" name="development_settings_enabled" value="1" package="com.android.settings" defaultValue="1" defaultSysSet="true" preserve_in_restore="true" /> <setting id="10" name="window_animation_scale" value="1.0" package="android" defaultValue="1.0" defaultSysSet="true" /> <setting id="136" name="stylus_handwriting_enabled" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" /> <setting id="129" name="force_allow_on_external" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" /> <setting id="106" name="multi_sim_data_call" value="2" package="com.android.phone" defaultValue="2" defaultSysSet="true" preserve_in_restore="true" /> <setting id="122" name="tether_offload_disabled" value="1" package="com.android.settings" defaultValue="1" defaultSysSet="true" /> <setting id="105" name="battery_estimates_last_update_time" value="1668412978004" package="com.android.systemui" defaultValue="1668412978004" defaultSysSet="true" preserve_in_restore="true" /> <setting id="9" name="mode_ringer" value="2" package="android" defaultValue="2" defaultSysSet="true" /> <setting id="36" name="default_install_location" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="22" name="wifi_display_on" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="44" name="device_name" value="ASUS_I006D" package="android" defaultValue="ASUS_I006D" defaultSysSet="true" /> <setting id="13" name="wifi_networks_available_notification_on" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="29" name="dock_sounds_enabled_when_accessbility" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="14" name="bluetooth_on" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="115" name="bugreport_in_power_menu" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" /> <setting id="18" name="mobile_data" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="160" name="network_watchlist_last_report_time" value="1668898800000" package="android" defaultValue="1668898800000" defaultSysSet="true" preserve_in_restore="true" /> <setting id="135" name="autofill_logging_level" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" /> <setting id="46" name="database_creation_buildid" value="SQ3A.220605.009.A1" package="android" defaultValue="SQ3A.220605.009.A1" defaultSysSet="true" /> <setting id="118" name="art_verifier_verify_debuggable" value="1" package="com.android.settings" defaultValue="1" defaultSysSet="true" /> <setting id="37" name="emergency_tone" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="114" name="adb_allowed_connection_time" value="604800000" package="com.android.settings" defaultValue="604800000" defaultSysSet="true" /> <setting id="53" name="max_sound_trigger_detection_service_ops_per_day" value="1000" package="android" defaultValue="1000" defaultSysSet="true" /> <setting id="47" name="add_users_when_locked" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="34" name="dock_audio_media_enabled" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="117" name="enable_gpu_debug_layers" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" /> <setting id="143" name="adb_enabled" value="1" package="com.android.settings" defaultValue="1" defaultSysSet="true" preserve_in_restore="true" /> <setting id="25" name="trusted_sound" value="/product/media/audio/ui/Trusted.ogg" package="android" defaultValue="/product/media/audio/ui/Trusted.ogg" defaultSysSet="true" /> <setting id="24" name="unlock_sound" value="/product/media/audio/ui/omni_unlock_phone.ogg" package="android" defaultValue="/product/media/audio/ui/omni_unlock_phone.ogg" defaultSysSet="true" /> <setting id="130" name="force_resizable_activities" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" /> <setting id="2" name="airplane_mode_radios" value="cell,bluetooth,wifi,nfc,wimax" package="android" defaultValue="cell,bluetooth,wifi,nfc,wimax" defaultSysSet="true" /> <setting id="21" name="wifi_max_dhcp_retry_count" value="9" package="android" defaultValue="9" defaultSysSet="true" /> <setting id="126" name="always_finish_activities" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="3" name="airplane_mode_toggleable_radios" value="bluetooth,wifi,nfc" package="android" defaultValue="bluetooth,wifi,nfc" defaultSysSet="true" /> <setting id="43" name="heads_up_notifications_enabled" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="78" name="low_power" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="58" name="charging_started_sound" value="/product/media/audio/ui/ChargingStarted.ogg" package="android" defaultValue="/product/media/audio/ui/ChargingStarted.ogg" defaultSysSet="true" /> <setting id="64" name="zen_mode_ringer_level" value="2" package="android" defaultValue="2" defaultSysSet="true" /> <setting id="65" name="user_disabled_hdr_formats" value="" package="android" defaultValue="" defaultSysSet="true" /> <setting id="39" name="preferred_network_mode" value="26,26" package="android" defaultValue="26,26" defaultSysSet="true" /> <setting id="48" name="bluetooth_disabled_profiles" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="31" name="desk_undock_sound" value="/product/media/audio/ui/Undock.ogg" package="android" defaultValue="/product/media/audio/ui/Undock.ogg" defaultSysSet="true" /> <setting id="55" name="zen_duration" package="android" /> <setting id="121" name="mobile_data_always_on" value="1" package="com.android.settings" defaultValue="1" defaultSysSet="true" preserve_in_restore="true" /> <setting id="97" name="debug_app" package="android" preserve_in_restore="true" /> <setting id="8" name="wifi_sleep_policy" value="2" package="android" defaultValue="2" defaultSysSet="true" /> <setting id="28" name="dock_sounds_enabled" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="107" name="wifi_scan_always_enabled" value="1" package="android" defaultValue="1" defaultSysSet="true" preserve_in_restore="true" /> <setting id="127" name="cached_apps_freezer" value="device_default" package="com.android.settings" defaultValue="device_default" defaultSysSet="true" /> <setting id="125" name="overlay_display_devices" package="com.android.settings" /> <setting id="87" name="data_roaming1" value="0" package="com.android.phone" defaultValue="0" defaultSysSet="true" /> <setting id="84" name="data_roaming2" value="0" package="com.android.phone" defaultValue="0" defaultSysSet="true" /> <setting id="138" name="enable_back_animation" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" /> <setting id="82" name="network_recommendations_enabled" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="133" name="enable_non_resizable_multi_window" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" preserve_in_restore="true" /> <setting id="102" name="bluetooth_sanitized_exposure_notification_supported" value="1" package="com.android.bluetooth" defaultValue="1" defaultSysSet="true" preserve_in_restore="true" /> <setting id="32" name="car_dock_sound" value="/product/media/audio/ui/Dock.ogg" package="android" defaultValue="/product/media/audio/ui/Dock.ogg" defaultSysSet="true" /> <setting id="54" name="sound_trigger_detection_service_op_timeout" value="15000" package="android" defaultValue="15000" defaultSysSet="true" /> <setting id="79" name="updatable_driver_production_denylist" value="" package="android" defaultValue="" defaultSysSet="true" /> <setting id="26" name="power_sounds_enabled" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="111" name="stay_on_while_plugged_in" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" preserve_in_restore="true" /> <setting id="104" name="average_time_to_discharge" value="-1" package="com.android.systemui" defaultValue="-1" defaultSysSet="true" preserve_in_restore="true" /> <setting id="45" name="lid_behavior" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="159" name="zen_mode_config_etag" value="1249128678" package="android" defaultValue="1249128678" defaultSysSet="true" preserve_in_restore="true" /> <setting id="35" name="set_install_location" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="116" name="debug_view_attributes" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" /> <setting id="62" name="zen_mode" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="11" name="transition_animation_scale" value="1.0" package="android" defaultValue="1.0" defaultSysSet="true" /> <setting id="16" name="data_roaming" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="80" name="sysui_tuner_demo_on" value="0" package="com.android.systemui" defaultValue="0" defaultSysSet="true" /> <setting id="123" name="debug.force_rtl" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" /> <setting id="1" name="theater_mode_on" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="15" name="cdma_cell_broadcast_sms" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="137" name="ingress_rate_limit_bytes_per_second" value="-1" package="com.android.settings" defaultValue="-1" defaultSysSet="true" /> <setting id="0" name="airplane_mode_on" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="132" name="force_desktop_mode_on_external_displays" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" /> <setting id="50" name="default_restrict_background_data" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="27" name="low_battery_sound" value="/product/media/audio/ui/omni_lowbattery1.ogg" package="android" defaultValue="/product/media/audio/ui/omni_lowbattery1.ogg" defaultSysSet="true" /> <setting id="56" name="apply_ramping_ringer" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="95" name="wait_for_debugger" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="19" name="netstats_enabled" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="49" name="wifi_wakeup_enabled" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="112" name="ota_disable_automatic_update" value="1" package="com.android.settings" defaultValue="1" defaultSysSet="true" /> <setting id="63" name="wifi_migration_completed" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="101" name="time_remaining_estimate_millis" value="-1" package="com.android.systemui" defaultValue="-1" defaultSysSet="true" preserve_in_restore="true" /> <setting id="40" name="subscription_mode" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="4" name="assisted_gps_enabled" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="23" name="lock_sound" value="/product/media/audio/ui/omni_lock_phone.ogg" package="android" defaultValue="/product/media/audio/ui/omni_lock_phone.ogg" defaultSysSet="true" /> <setting id="120" name="non_persistent_mac_randomization_force_enabled" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" /> <setting id="6" name="auto_time_zone" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="134" name="enable_gnss_raw_meas_full_tracking" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" /> <setting id="88" name="wifi_on" value="1" package="android" defaultValue="1" defaultSysSet="true" preserve_in_restore="true" /> <setting id="57" name="wireless_charging_started_sound" value="/product/media/audio/ui/WirelessChargingStarted.ogg" package="android" defaultValue="/product/media/audio/ui/WirelessChargingStarted.ogg" defaultSysSet="true" /> <setting id="83" name="device_provisioned" value="1" package="com.android.provision" defaultValue="1" defaultSysSet="true" preserve_in_restore="true" /> <setting id="119" name="wifi_display_certification_on" value="0" package="com.android.settings" defaultValue="0" defaultSysSet="true" /> <setting id="81" name="sysui_demo_allowed" value="0" package="com.android.systemui" defaultValue="0" defaultSysSet="true" /> <setting id="59" name="hdmi_control_auto_device_off_enabled" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="5" name="auto_time" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="20" name="usb_mass_storage_enabled" value="1" package="android" defaultValue="1" defaultSysSet="true" /> <setting id="38" name="call_auto_retry" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="77" name="audio_safe_volume_state" value="3" package="android" defaultValue="3" defaultSysSet="true" /> <setting id="141" name="boot_count" value="4" package="android" defaultValue="4" defaultSysSet="true" preserve_in_restore="true" /> <setting id="30" name="desk_dock_sound" value="/product/media/audio/ui/Dock.ogg" package="android" defaultValue="/product/media/audio/ui/Dock.ogg" defaultSysSet="true" /> <setting id="124" name="animator_duration_scale" value="1.0" package="android" defaultValue="1.0" defaultSysSet="true" /> </settings> ASUS_I006D:/ #

Working with files in Android Binary XML format on a PC

I do not know a tool to convert a file in Android Binary XML format to human readable format on a PC.

There is a tool to convert binary XML files used in apk files in the AMXL format to human readable format and vice versa called xml2axml (https://github.com/hzw1199/xml2axml)
XML files used in apk files are in AMXL format, which is also a binary format for XML files, but different from the Android Binary XML format.
I used xml2axml successfully to convert XML files used in apk files but xml2axml does not work for files in the Android Binary XML format.

I did not test it but I assume that's also true for other tools to work with apk files like apktool or dex2jar.

Other tools for files in Android Binary XML format I found are:

The abx converter from here https://github.com/cclgroupltd/android-bits/tree/main/ccl_abx claims to be a tool to convert Android Binary XML files but seems not to work for Android Binary XML files from OmniROM 13:

[xtrnaw7@t15g /data/develop/android/android-bits-main/ccl_abx]$ python3 ccl_abx.py ../../test2/settings_global.xml Traceback (most recent call last): File "/data/develop/android/android-bits-main/ccl_abx/ccl_abx.py", line 292, in <module> main(sys.argv[1:]) File "/data/develop/android/android-bits-main/ccl_abx/ccl_abx.py", line 284, in main doc = reader.read(is_multi_root=multi_root) File "/data/develop/android/android-bits-main/ccl_abx/ccl_abx.py", line 266, in read raise NotImplementedError(f"unexpected XML type: {xml_type}") NotImplementedError: unexpected XML type: 7 [xtrnaw7@t15g /data/develop/android/android-bits-main/ccl_abx]$

Update 23.11.2022 /bs

ccl_abx.py works for the Android Binary XML files from OmniROM 12, e.g.

[xtrnaw7@t15g /data/develop/android/android-bits-main/ccl_abx]$ python3 ccl_abx.py ../../test2/settings_global_a12.xml -mr | sed "s/>/>\n/g" | more <root> <settings version="205"> <setting id="67" name="adb_wifi_enabled" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="91" name="mobile_data1" value="0" package="com.android.phone" defaultValue="0" defaultSysSet="true" preserve_in_restore="true" /> <setting id="87" name="mobile_data2" value="0" package="com.android.phone" defaultValue="0" defaultSysSet="true" /> <setting id="41" name="low_battery_sound_timeout" value="0" package="android" defaultValue="0" defaultSysSet="true" /> <setting id="33" name="car_undock_sound" value="/product/media/audio/ui/Undock.ogg" package="android" defaultValue="/product/media/audio/ui/Undock.ogg" defaultSysSet="true" /> <setting id="101" name="time_remaining_estimate_based_on_usage" value="0" package="com.android.systemui" defaultValue="0" defaultSysSet="true" preserve_in_restore="true" /> .... <setting id="85" name="wifi_on" value="1" package="android" defaultValue="1" defaultSysSet="true" preserve_in_restore="true" /> <setting id="57" name="wireless_charging_started_sound" value="/product/media/audio/ui/WirelessChargingStarted.ogg" package="android" defaultValue="/product/media/audio/ui/WirelessChargingStarted.ogg" defaultSysSet="true" /> [xtrnaw7@t15g /data/develop/android/android-bits-main/ccl_abx]$

There is a github repo with an abx parser in go https://github.com/SagerNet/abx-go (I did not test this)

Example:

I wrote a little script to use the method described above to enable WiFi in the persistent settings:

enable_wifi.sh

The script will enable WiFi in the persistent settings if currently disabled; if WiFi is already enabled the script will change nothing.

The script must be executed by the user root on the phone.

Example output of the script

Output if WiFi is disabled:

ASUS_I006D:/ # /data/local/tmp/enable_wifi.sh Checking WiFi status in the persistent global settings ... Converting the file "/data/system/users/0/settings_global.xml" to an XML file in text format "data/local/tmp/setttings_global.xml.human.9574" ... WiFi is currently disabled -- enabling WiFi now :.. ... WiFi enabled in the persistent global settings. To activate the change reboot the phone ASUS_I006D:/ #
Output if Wifi is already enabled:

ASUS_I006D:/ # /data/local/tmp/enable_wifi.sh Checking WiFi status in the persistent global settings ... Converting the file "/data/system/users/0/settings_global.xml" to an XML file in text format "data/local/tmp/setttings_global.xml.human.9613" ... WiFi is already enabled in the persistent global settings ASUS_I006D:/ #

The script is attached to this post ; it can also be downloaded from here: enable_wifi.sh

Important Note

After writing this HowTo I discovered that there is a native Android command to change the settings stored in the file /data/system/users/0/settings_global.xml:

settings

settings usage

ASUS_I006D:/ $ settings Settings provider (settings) commands: help Print this help text. get [--user <USER_ID> | current] NAMESPACE KEY Retrieve the current value of KEY. put [--user <USER_ID> | current] NAMESPACE KEY VALUE [TAG] [default] Change the contents of KEY to VALUE. TAG to associate with the setting. {default} to set as the default, case-insensitive only for global/secure namespace delete [--user <USER_ID> | current] NAMESPACE KEY Delete the entry for KEY. reset [--user <USER_ID> | current] NAMESPACE {PACKAGE_NAME | RESET_MODE} Reset the global/secure table for a package with mode. RESET_MODE is one of {untrusted_defaults, untrusted_clear, trusted_defaults}, case-insensitive list [--user <USER_ID> | current] NAMESPACE Print all defined keys. NAMESPACE is one of {system, secure, global}, case-insensitive 255|ASUS_I006D:/ $

settings example

e.g to set or get the WiFi status:

ASUS_I006D:/ $ settings get global wifi_on 1 ASUS_I006D:/ $ ASUS_I006D:/ $ settings put global wifi_on ASUS_I006D:/ $ ASUS_I006D:/ $ settings get global wifi_on 0 ASUS_I006D:/ $ ASUS_I006D:/ $ settings put global wifi_on ASUS_I006D:/ $ ASUS_I006D:/ $ settings get global wifi_on 1 ASUS_I006D:/ $

A description of the known settings is here:

https://developer.android.com/reference/android/provider/Settings.Global
https://developer.android.com/reference/android/provider/Settings.Secure
https://developer.android.com/reference/android/provider/Settings.System

But since this HowTo is about editing files in Android Binary XML format and the change of the Wifi status is only meant as an example, I leave it like this.

How to compile TWRP for the ASUS Zenfone 8

URL: https://xdaforums.com/t/how-to-compile-twrp-for-the-asus-zenfone-8.4523817/

How to compile TWRP for the ASUS Zenfone 8

There are a lot of web pages about building your own TWRP image in the internet but at least I did not find a HowTo with all necessary instructions and infos in one place.

Therefor I wrote this little HowTo about building a TWRP image for the ASUS Zenfone 8. The instructions in the HowTo can also be used to create TWRP images for other phones supported by TWRP.

Note that this HowTo can only be used for phones that are officially supported by TWRP. The HowTo is for a PC running the Linux OS.

In the end, creating a TWRP image is also quite easy once you know how it works, thanks to the well-done setup of the development environment by the TWRP developer.

Prereqs

The repositories with the manifests for TWRP are here https://github.com/minimal-manifest-twrp and the repositories with the TWRP files and the device trees for the phones supported by TWRP are here: https://github.com/TeamWin. So you need access to github for creating your own TWRP image.

The tool repo is required to maintain the repositories for TWRP; you can download repo from https://gerrit.googlesource.com/git-repo

It's not necessary to install additional tools for creating an TWRP image -- all necessary tools to create the image are in the repositories for TWRP.

Note

To check if there is a repository with the device tree for another phone for TWRP use the search function on the page https://github.com/TeamWin and search for the device ID of the phone.

To search via script use (I006D is the device ID for the ASUS Zenfone 8):

wget "https://api.github.com/search/repositories?q=I006D" -O ./repository_list

search example for the ASUS Zenfone 8

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrpX ] $ wget "https://api.github.com/search/repositories?q=I006D" -O ./repository_list --2022-11-27 09:20:27-- https://api.github.com/search/repositories?q=I006D Resolving api.github.com (api.github.com)... 140.82.121.5 Connecting to api.github.com (api.github.com)|140.82.121.5|:443... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [application/json] Saving to: â€˜./repository_listâ€™ ./repository_list [ <=> ] 46.30K --.-KB/s in 0.01s 2022-11-27 09:20:28 (3.55 MB/s) - â€˜./repository_listâ€™ saved [47414] [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrpX ] $ [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrpX ] $ grep "full_name" ./repository_list "full_name": "CaptainThrowback/android_device_asus_I006D", "full_name": "nguyenhung9xdev2022/android_kernel_asus_I006D", "full_name": "nguyenhung9xdev2022/android_vendor_asus_I006D", "full_name": "nguyenhung9xdev2022/android_device_asus_I006D", "full_name": "asus-development/android_device_asus_I006D", "full_name": "OpenPecha-Data/I006D4C7E", "full_name": "dmd79/android_device_asus_I006D", [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrpX ] $

To get the device ID for a phone execute :

adb shell getprop ro.product.system.device

e,g.

# # on a PC for a connected ASUS Zenfone 8 # [xtrnaw7@t15g /data/develop/android/twrp]$ adb shell getprop ro.product.system.device I006D [xtrnaw7@t15g /data/develop/android/twrp]$ # # on the ZUK Z2 Pro # z2_row:/ # getprop ro.product.system.device z2_row z2_row:/ #

The device ID is required for the lunch command used to copy the repositories with the device trees (see below).

Building the ROM

First copy the necessary repositories for building TWRP to your local machine using these commands:

# # create a new directory for the repositories (optional) # mkdir twrp cd twrp # init the local repositories # # As of 27.11.2022 the latest version of the TWRP repositories is 12.1 ; # to download the default repository version you can omit the parameter "-b twrp-12.1" # As of 27.11.2022 the repositories for TWRP 12.1 use about 45 GB; # to save some space (about 12 GB) you could add the parameter --depth=1 # repo init -u https://github.com/minimal-manifest-twrp/platform_manifest_twrp_aosp.git -b twrp-12.1 # sync the repositories (Parameter-j8 = use 8 parallel jobs; adjust this parameter to the number of CPU available on your PC) # repo sync -c -j8 --force-sync --no-clone-bundle --no-tags

Next add the device dependent repositories for the ASUS Zenfone 8.

The format of the parameter for the lunch command is

twrp_<device_id>-eng

So the parameter for the ASUS Zenfone 8 is twrp_I006D-eng.

# # init the environment to build the TWRP image # . build/envsetup.sh export ALLOW_MISSING_DEPENDENCIES=true lunch twrp_I006D-eng

lunch example

TWRP Dev - xtrnaw7@t15g /data/develop/android/twrp ] $
          lunch twrp_I006D-eng

          In file included from build/make/core/config.mk:313:

          In file included from build/make/core/envsetup.mk:312:

          build/make/core/product_config.mk:160: error: Can not locate
          config makefile for product "twrp_I006D".

          11:35:37 dumpvars failed with: exit status 1

          Device I006D not found. Attempting to retrieve device
          repository from TeamWin Github (http://github.com/TeamWin).

          Found repository: android_device_asus_I006D

          Checking branch info

          Calculated revision: android-12.1

          Adding dependency:

          Repository: android_device_asus_I006D

          Branch: android-12.1

          Remote: TeamWin

          Path: device/asus/I006D

          

          Syncing repository to retrieve project.

          Fetching: 100% (1/1), done in 12.846s

          NOT Garbage collecting: 0% (0/1), done in 0.001s

          repo sync has finished successfully.

          Repository synced!

          Looking for dependencies

          Adding dependencies to manifest

          

          Adding dependency:

          Repository: android_device_asus_sm8350-common

          Branch: android-12.1

          Remote: TeamWin

          Path: device/asus/sm8350-common

          

          Syncing dependencies

          Fetching: 100% (1/1), done in 15.848s

          NOT Garbage collecting: 0% (0/1), done in 0.001s

          repo sync has finished successfully.

          Looking for dependencies

          device/asus/sm8350-common has no additional dependencies.

          Done

          

          ============================================

          PLATFORM_VERSION_CODENAME=REL

          PLATFORM_VERSION=99.87.36

          TARGET_PRODUCT=twrp_I006D

          TARGET_BUILD_VARIANT=eng

          TARGET_BUILD_TYPE=release

          TARGET_ARCH=arm64

          TARGET_ARCH_VARIANT=armv8-a

          TARGET_CPU_VARIANT=generic

          TARGET_2ND_ARCH=arm

          TARGET_2ND_ARCH_VARIANT=armv8-a

          TARGET_2ND_CPU_VARIANT=generic

          HOST_ARCH=x86_64

          HOST_2ND_ARCH=x86

          HOST_OS=linux

HOST_OS_EXTRA=Linux-6.0.9-200.fc36.x86_64-x86_64-Fedora-Linux-36-(MATE-Compiz)

          HOST_CROSS_OS=windows

          HOST_CROSS_ARCH=x86

          HOST_CROSS_2ND_ARCH=x86_64

          HOST_BUILD_TYPE=release

          BUILD_ID=SP2A.220405.004

          OUT_DIR=out

          PRODUCT_SOONG_NAMESPACES=device/asus/sm8350-common

          ============================================

          [ TWRP Dev - xtrnaw7@t15g /data/develop/android/twrp ] $

And final build the TWRP image

mka bootimage

mka bootimage example

[ TWRP Dev - xtrnaw7@t15g /data/develop/android/twrp ] $
          time mka bootimage

          build/make/core/soong_config.mk:197: warning:
          BOARD_PLAT_PUBLIC_SEPOLICY_DIR has been deprecated. Use
          SYSTEM_EXT_PUBLIC_SEPOLICY_DIRS instead.

          build/make/core/soong_config.mk:198: warning:
          BOARD_PLAT_PRIVATE_SEPOLICY_DIR has been deprecated. Use
          SYSTEM_EXT_PRIVATE_SEPOLICY_DIRS instead.

          ============================================

          PLATFORM_VERSION_CODENAME=REL

          PLATFORM_VERSION=99.87.36

          TARGET_PRODUCT=twrp_I006D

          TARGET_BUILD_VARIANT=eng

          TARGET_BUILD_TYPE=release

          TARGET_ARCH=arm64

          TARGET_ARCH_VARIANT=armv8-a

          TARGET_CPU_VARIANT=generic

          TARGET_2ND_ARCH=arm

          TARGET_2ND_ARCH_VARIANT=armv8-a

          TARGET_2ND_CPU_VARIANT=generic

          HOST_ARCH=x86_64

          HOST_2ND_ARCH=x86

          HOST_OS=linux

HOST_OS_EXTRA=Linux-6.0.9-200.fc36.x86_64-x86_64-Fedora-Linux-36-(MATE-Compiz)

          HOST_CROSS_OS=windows

          HOST_CROSS_ARCH=x86

          HOST_CROSS_2ND_ARCH=x86_64

          HOST_BUILD_TYPE=release

          BUILD_ID=SP2A.220405.004

          OUT_DIR=out

          PRODUCT_SOONG_NAMESPACES=device/asus/sm8350-common

          ============================================

          [ 78% 246/312] including bootable/recovery/Android.mk ...

          bootable/recovery/prebuilt/Android.mk:437: warning: vendor_hw:
          bootable/recovery/prebuilt/relink.sh
          out/target/product/I006D/recovery/root/vendor/bin/hw

          [ 95% 299/312] including system/sepolicy/Android.mk ...

          system/sepolicy/Android.mk:57: warning:
          BOARD_PLAT_PUBLIC_SEPOLICY_DIR has been deprecated. Use
          SYSTEM_EXT_PUBLIC_SEPOLICY_DIRS instead.

          system/sepolicy/Android.mk:62: warning:
          BOARD_PLAT_PRIVATE_SEPOLICY_DIR has been deprecated. Use
          SYSTEM_EXT_PRIVATE_SEPOLICY_DIRS instead.

          [ 30% 8637/28297] //bionic/libc:libc_bionic clang
          arch-arm64/bionic/setjmp.S

          ....

          Removing unneeded service: keymaster-4-1-citadel

          

          Service_Cleanup script complete.

          

          [100% 28297/28297] Target boot image from recovery:
          out/target/product/I006D/boot.img

          

          #### build completed successfully (14:50 (mm:ss)) ####

          

          

          real 14m50.110s

          user 182m6.217s

          sys 11m58.492s

          [ TWRP Dev - xtrnaw7@t15g /data/develop/android/twrp ] $

Note:

For devices without A/B devices use mka recoveryimage.

The TWRP image will be created in file boot.img in the directory ./out/target/product/I006D, e.g.:

[ TWRP Dev - xtrnaw7@t15g /data/develop/android/twrp ] $ ls -ltr out/target/product/I006D/boot.img -rw-rw-r--. 1 xtrnaw7 xtrnaw7 100663296 Nov 27 12:06 out/target/product/I006D/boot.img [ TWRP Dev - xtrnaw7@t15g /data/develop/android/twrp ] $

To test the image just boot the phone from the image

adb reboot bootloader sudo fastboot boot out/target/product/I006D/boot.img

And check if the boot was succesfull:

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrpX ] $ adb shell getprop ro.bootmode unknown [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrpX ] $
unknown is the expected value here.

Trouble Shooting

On some pages I found a hint that the build scripts require python2 and will not work with python3. But this is most probably not true anymore because building the image with python3 worked without problems.

It's strongly recommended to use separate sessions for compiling OmniROM or TWRP or even different versions of TWRP:

All these build environments define the same commands in the file ./build/envsetup.sh but with different functionality.

As of 27.11.2022 there are only 2 branches in the repository with the manifests for TWRP : version 11 and version 12.1.
If there is no branch for one of these versions in the repositories for the device tree of the phone for which you want to create a TWRP image the lunch command will fail.

The repositories with the manifests for TWRP prior to version 11 can be fetched with this command:

# Minimal manifest for building TWRP for devices shipped with Android 5.1 through Android 9.0 # repo init -u https://github.com/minimal-manifest-twrp/platform_manifest_twrp_omni.git -b twrp-9.0

Note that the lunch command in the old repositories does work different

Notes:

The URL to the repository with the device tree for a phone is also on the page for the device on the TWRP device list: https://twrp.me/Devices/

See https://unofficialtwrp.com/build-compile-twrp-recovery/ for other possible settings for the build

see here for How to add additional files to an TWRP imagesee also How to compile the image for TWRP 3.7.0.12-1 w/ the bootctl binary

XDA Forum thread about compiling TWRP: How to compile TWRP touch recovery

How to compile the image for TWRP 3.7.0.12-1 w/ the bootctl binary

URL: https://xdaforums.com/t/how-to-compile-the-image-for-twrp-3-7-0-12-1-w-the-bootctl-binary.4650684/

How to compile the image for TWRP 3.7.0.12-1 w/ the bootctl binary

In the official images for TWRP version 3.7.0.12-1 the binaries



          bootctl 

          lpdump 

          lpdumpd 

          lptools

are not included anymore - see here: https://github.com/TeamWin/android_bootable_recovery/commit/aa3c77a0b5364782ab734cd8c511b85798d7452d

But it's simple to create your own TWRP image that again contains these binaries:

The post How to compile TWRP for the ASUS Zenfone8 describes in general how to compile the TWRP image.

To compile a TWRP image for version 3.7.0.12-1 that contains the missing binaries, use these commands:

# make the directory with the TWRP source the working directory # cd /data/develop/android/twrp . build/envsetup.sh export ALLOW_MISSING_DEPENDENCIES=true # set this variable to include the missing binaries in the TWRP image # export TW_INCLUDE_LPDUMP=yes # create a TRWP image for the ASUS Zenfone 8 (change the parameter for lunch to match your phone!) # lunch twrp_I006D-eng mka bootimage

Some hints for using TWRP

URL: https://xdaforums.com/t/some-hints-using-twrp.4525833/

Some hints using TWRP

This HowTo contains some hints for using TWRP in general and some tips for using CLI commands in TWRP.

I will update this post in case I find something new & useful regarding TWRP and update the history section at the end of this post.

The test environment for the instructions below was

TWRP 3.6.2.12-0 for the ASUS Zenfone 8
TWRP 3.7.0_12-0 for the ASUS Zenfone 8

Note:

This is NOT a general HowTo for TWRP.

How to test a TWRP image?

It's recommended to boot from the TWRP image file before installing it into the boot or recovery partition of your phone. To do this do

# connect the phone via USB to your PC # boot the phone into the bootloader - e.g. via # adb reboot bootloader # boot the phone from the TWRP image (this will not change anything on the phone) # sudo fastboot boot <twrpimage_file>

If everything works you can install the TWRP image into the boot or recovery partition using the approbiate menu entry in TWRP.

How to connect via adb to an TWRP session?

TWRP supports adb session as user root, therefor an

adb shell

on your PC is sufficient to open an adb shell as user root on the phone booted into TWRP.

In case you get a permission denied error kill the running adb daemon on the PC and restart the adb command. If that does not fix the error disconnect and reconnect the USB cable between the PC and the Phone.

Be aware that TWRP starts the adb daemon twice after booting. Therefor an adb session to the phone started to early after the boot into TWRP might be killed. In this case just open a new adb shell.

How to check if the phone booted into TWRP ?

Check the value of the property ro.twrp.boot; this property is only set if TWRP is running

e.g:

ASUS_I006D:/ # getprop ro.twrp.boot1 ASUS_I006D:/ #

How to check if the running TWRP was booted from an TWRP image or from a boot or recovery partition?

Check the value of the property ro.bootmode; the value of this property is "unknown" if TWRP was booted from an image file via "fastboot boot <twrp_image_file>", e.g.

ASUS_I006D:/ # getprop ro.bootmode unknown ASUS_I006D:/ #

-> TWRP was booted from an TWRP image file

ASUS_I006D:/ # getprop ro.bootmode recovery ASUS_I006D:/ #

-> TWRP was booted from the boot or recovery partition

How to get the version of the running TWRP?

Check the value of the property getprop ro.twrp.version, e.g.:

ASUS_I006D:/ # getprop ro.twrp.version 3.7.0_12-0 ASUS_I006D:/ #

Where is the log file from TWRP?

The logfile with the messages written while booting TWRP is /data/recovery/log.gz.

e.g.

130|ASUS_I006D:/ # ls -ltr /data/recovery/l*gz -rw------- 1 system system 8022 2022-12-26 14:03 /data/recovery/log.gz ASUS_I006D:/ # ASUS_I006D:/ # gzip -cd /data/recovery/log.gz | head -10Starting TWRP 3.7.0_12-0-d07fdb3c on Sat Jan 10 00:53:15 1970 (pid 403) I:Lun file '/config/usb_gadget/g1/functions/mass_storage.0/lun.0/file' PRODUCT_USE_DYNAMIC_PARTITIONS := true TW_INCLUDE_CRYPTO := true I:TW_BRIGHTNESS_PATH := /proc/lcd_brightness I:Found brightness file at '/proc/lcd_brightness' I:TWFunc::Set_Brightness: Setting brightness control to 420 I:TW_EXCLUDE_ENCRYPTED_BACKUPS := true I:LANG: en ASUS_I006D:/ #

The current logfile of the running TWRP is /tmp/recovery.log, e.g.:

ASUS_I006D:/ # ls -l /tmp/recovery.log -rw-rw-rw- 1 root root 43303 2023-05-19 20:11 /tmp/recovery.log ASUS_I006D:/ #

Where are the settings from TWRP stored?

The settings from TWRP are stored in the file

/data/media/0/TWRP/.twrps

This is a binary file.

Note that on phones with active encryption that file is only accessible after /data is decrypted.

How to edit a file while booted into TWRP?

The editor nano is part of TWRP and can be used to edit files in an adb session.

How to mount a partition while inside TWRP?

Use

twrp mount <mountpoint>

to mount a partition while TWRP is running.

Note that you should use this command with caution in adb sessions because the adb session may be killed after the mount

How to decrypt an encrypted data partition form within a booted TWRP

Use

twrp decrypt <password>

to decrypt an encrypted /data partition, e.g.:

ASUS_I006D:/ # ls /data/media/0+lnodCAAAAAnYFLObBoWS5ZNb5yVMpze 8KdHMCAAAAQOgtDHq,XdSfFPW+XURA2M KlfTJCAAAAwL93jkH2ugnU4eUG3Zzm7M W9W9SCAAAAgh6lwoP0mNiEK9VIWJoH6L 0nUvhAAAAAQ9ptz+L5qPQrpzcAt4G06K DWk4KDAAAAA,B1xVDmT3dZI87g2C0IS8 M6jcmCAAAAAU,4gn9o0zoBZmkysyXy49 bE7t0CAAAAQ2YuHL9styNaf,iSf2cCO, 4nFjMBAAAAwv2oUczXEpKUDfRIOi+nR5 EcHrdAAAAAglBtbhNWl0NvyX8aW+3CHN O5ZVrDAAAAAlKd7PPpo7CaQmkFbZzKRm haPcnBAAAAw9o2lLF97zpPTaC+byP92s 5yL8gCAAAAgekBDBUi4,W78rJAkgFcMF FA1aZAAAAAg2XNF+M859lpPY,3rUC2cY Vda+LCAAAAwR+g6Tav35+h,Epc8latGR mSKc8BAAAAwm+12sMClHPf+zwtHAeW+Q 8CsF+DAAAAwI3tImfOqIS9aHCQ3S7Cdg JpuFgAAAAAwdEzuQerx4681dzGLjlK3X VvEV4DAAAAgRgoqCfrypNqryZxRACnfr rMJEjAAAAAgwTgx,vA5ewWMVizc2LAos ASUS_I006D:/ # ASUS_I006D:/ # twrp decrypt 12345Attempting to decrypt data partition or user data via command line. Attempting to decrypt FBE for user 0... User 0 Decrypted Successfully Data successfully decrypted Updating partition details... ...done ASUS_I006D:/ # ASUS_I006D:/ # ls /data/media/0Alarms Android Audiobooks DCIM Documents Download Movies Music Notifications Pictures Podcasts Recordings Ringtones TWRP ASUS_I006D:/ #

see the TWRP Commandline Guide for the known parameter for this command

Note that existing adb sessions will be stopped after sucessfully executing this command.

How to check if the /data partition is encrypted from within a booted TWRP

To check if the data partition is encrypted in a script executed in a running TWRP use code like this:

PASSWORD="1234" CRYPTO_READY=$( getprop crypto.ready ) TWRP_DECRYPT_DONE=$( getprop twrp.decrypt.done ) if [ "${TWRP_DECRYPT_DONE}"x = "true"x ] ; then echo "The data partition is not encrypted or already decrypted" elif [ "${CRYPTO_READY}"x = "1"x ] ; then echo "The data partition is encrypted" # sample commands to decrypt the data partition # # echo "Decryptiong the data partition now ..." # twrp decrypt "${PASSWORD}" elif [ "${CRYPTO_READY}"x = "0"x ] ; then echo "The data partition is not encrypted" else echo "ERROR: The value of the property crypto.ready \"${CRYPTO_READY}\" is not known" fi

How to install a ZIP file via CLI command from within a running TWRP?

Copy the zip file to the phone via adb push and then use

twrp install /path/to/update.zip

to install a ZIP file from within TWRP

Note that the parameter must be a fully qualified filename.

Using this method to update the OS only works in a terminal session on the phone; it does not work in a adb session (at least in my tests until now ...)

What commands are available in TWRP?

Most of the Unix commands are available in TWRP:

ls /bin

ASUS_I006D:/ # ls /bin[ env load_policy pigz swapon acpi erase_image log pkill sync adbd expand logcat pmap sysctl android.hardware.boot@1.0-service expr logd prepdecrypt.sh tac android.hardware.boot@1.1-service fallocate logname printenv tail android.hardware.boot@1.2-service false losetup printf tar android.hardware.gatekeeper@1.0-service-qti fastbootd lpdump privapp-permissions-twrpapp.xml taskset android.hardware.health@2.0-service fatlabel lpdumpd ps tee android.hardware.health@2.1-service fgrep lptools pwd test android.hardware.keymaster@4.1-service-qti file ls qcom_decrypt-timestamp time avbctl find lsattr qcom_decrypt_fbe-timestamp timeout awk flash_image lsmod qseecomd toolbox base64 flock lsof readelf top basename fmt lspci readlink touch bash free lsusb realpath toybox bash_twrp-timestamp fsck.exfat magiskboot reboot tr bc fsck.f2fs make_f2fs recovery true blockdev fsck.fat md5sum renice truncate bootctl fscryptpolicyget me.twrp.twrpapp.apk resetprop tty bu fsync microcom resize2fs tune2fs cal getconf minadbd restorecon twrp cat getenforce mkdir rm tzdata_twrp-timestamp charger getevent mke2fs rmdir ueventd chattr getprop mkexfatfs rmmod ulimit chcon grep mkfifo rtcwake umount chgrp groups mkfs.fat runcon uname chmod gunzip mknod sed uniq chown gzip mkswap sendevent unix2dos chroot head mktemp seq unlink chrt hostname modinfo servicemanager unpigz cksum hwclock modprobe setenforce unshare clear hwservicemanager more setprop unzip cmp i2cdetect mount setsid update_engine_sideload comm i2cdump mountpoint sgdisk uptime cp i2cget mv sh usleep cpio i2cset nano sha1sum uudecode cut iconv nano_twrp-timestamp sha224sum uuencode date id nc sha256sum uuidgen dd ifconfig netcat sha384sum vmstat debuggerd init netstat sha512sum vndservicemanager devmem inotifyd nice simg2img vold_prepare_subdirs df insmod nl sleep watch diff install nohup sload_f2fs watchdogd dirname ionice nproc sort wc dmesg iorenice nsenter split which dos2unix keystore2 od start whoami du keystore_cli_v2 orsin stat xargs dump_image kill orsout stop xxd e2fsck killall paste strace yes e2fsdroid linker64 patch strings zcat echo linker_asan64 pgrep stty zipinfo egrep ln pidof swapoff ziptool ASUS_I006D:/ #

Be aware that a lot of Unix commands in TWRP are implemented using a general binary called toybox (toybox is something like busybox) and do not support all parameter like the original Linux commands

ASUS_I006D:/ # ls -l /bin/ | wc -l261 ASUS_I006D:/ # ls -l /bin/ | grep toybox | wc -l175 ASUS_I006D:/ #

What other TRWP specific commands can be used while in a running TWRP?

twrp and a few others like bootctl (see below) or magiskboot (see below).

twrp usage

ASUS_I006D:/ # twrp TWRP openrecoveryscript command line tool, TWRP version 3.7.0_12-0 Allows command line usage of TWRP via openrecoveryscript commands. Some common commands include: install /path/to/update.zip backup <SDCRBAEM> [backupname] restore <SDCRBAEM> [backupname] wipe <partition name> format data sideload set <variable> [value] decrypt <password> [USER ID] remountrw fixperms mount <path> unmount <path> listmounts print <value> mkdir <directory> reboot [recovery|poweroff|bootloader|download|edl] See more documentation athttps://twrp.me/faq/openrecoveryscript.html ASUS_I006D:/ #

see also How to install an OS image using the TWRP binary twrp?

Can I use the twrp binary in my Android OS or another recovery?

The twrp executable does not really do all the work : it just writes the commands to the fifo /system/bin/orsout and reads the result from the fifo /system/bin/orsin. The real work is done by the TWRP daemon. Without the running TWRP daemon the twrp executable does not work. The TWRP daemon does not work in the Android OS and therefor it useless to copy the twrp executable to the Android OS.

How to change the active slot for the next reboot while in TWRP?

Use bootctl, e.g.
ASUS_I006D:/ # bootctl get-current-slot 1 ASUS_I006D:/ # bootctl set-active-boot-slot 0

The usage for the command bootctl is

bootctl usage

ASUS_I006D:/ # /bin/bootctl /bin/bootctl - command-line wrapper for the boot HAL. Usage: /bin/bootctl COMMAND Commands: hal-info - Show info about boot_control HAL used. get-number-slots - Prints number of slots. get-current-slot - Prints currently running SLOT. mark-boot-successful - Mark current slot as GOOD. get-active-boot-slot - Prints the SLOT to load on next boot. set-active-boot-slot SLOT - On next boot, load and execute SLOT. set-slot-as-unbootable SLOT - Mark SLOT as invalid. is-slot-bootable SLOT - Returns 0 only if SLOT is bootable. is-slot-marked-successful SLOT - Returns 0 only if SLOT is marked GOOD. get-suffix SLOT - Prints suffix for SLOT. set-snapshot-merge-status STAT - Sets whether a snapshot-merge of any dynamic partition is in progress. Valid STAT values are: none, unknown, snapshotted, merging, or cancelled. get-snapshot-merge-status - Prints the current snapshot-merge status. SLOT parameter is the zero-based slot-number. ASUS_I006D:/ #

see also How to switch the active slot

How to use Magisk binaries while booted into TWRP

Magisk is another great tool for Android phones -- for details about Magisk see here:

https://github.com/topjohnwu/Magisk

The Magisk binary for manipulating the boot partition, magiskboot, is part of TWRP. This binary can be used to unpack and repack boot partitions or boot partition images, ramdisks, or compress/decompress files. The binary works without problems while booted into TWRP.

magiskboot usage

1|ASUS_I006D:/ #magiskboot MagiskBoot - Boot Image Modification Tool Usage: magiskboot <action> [args...] Supported actions: unpack [-n] [-h] <bootimg> Unpack <bootimg> to, if available, kernel, kernel_dtb, ramdisk.cpio, second, dtb, extra, and recovery_dtbo into current directory. If '-n' is provided, it will not attempt to decompress kernel or ramdisk.cpio from their original formats. If '-h' is provided, it will dump header info to 'header', which will be parsed when repacking. Return values: 0:valid 1:error 2:chromeos repack [-n] <origbootimg> [outbootimg] Repack boot image components from current directory to [outbootimg], or new-boot.img if not specified. If '-n' is provided, it will not attempt to recompress ramdisk.cpio, otherwise it will compress ramdisk.cpio and kernel with the same format as in <origbootimg> if the file provided is not already compressed. If env variable PATCHVBMETAFLAG is set to true, all disable flags will be set in the vbmeta header. hexpatch <file> <hexpattern1> <hexpattern2> Search <hexpattern1> in <file>, and replace with <hexpattern2> cpio <incpio> [commands...] Do cpio commands to <incpio> (modifications are done in-place) Each command is a single argument, add quotes for each command Supported commands: exists ENTRY Return 0 if ENTRY exists, else return 1 rm [-r] ENTRY Remove ENTRY, specify [-r] to remove recursively mkdir MODE ENTRY Create directory ENTRY in permissions MODE ln TARGET ENTRY Create a symlink to TARGET with the name ENTRY mv SOURCE DEST Move SOURCE to DEST add MODE ENTRY INFILE Add INFILE as ENTRY in permissions MODE; replaces ENTRY if exists extract [ENTRY OUT] Extract ENTRY to OUT, or extract all entries to current directory test Test the current cpio's status Return value is 0 or bitwise or-ed of following values: 0x1:Magisk 0x2:unsupported 0x4:Sony patch Apply ramdisk patches Configure with env variables: KEEPVERITY KEEPFORCEENCRYPT backup ORIG Create ramdisk backups from ORIG restore Restore ramdisk from ramdisk backup stored within incpio sha1 Print stock boot SHA1 if previously backed up in ramdisk dtb <input> <action> [args...] Do dtb related actions to <input> Supported actions: print [-f] Print all contents of dtb for debugging Specify [-f] to only print fstab nodes patch Search for fstab and remove verity/avb Modifications are done directly to the file in-place Configure with env variables: KEEPVERITY split <input> Split image.*-dtb into kernel + kernel_dtb sha1 <file> Print the SHA1 checksum for <file> cleanup Cleanup the current working directory compress[=format] <infile> [outfile] Compress <infile> with [format] (default: gzip), optionally to [outfile] <infile>/[outfile] can be '-' to be STDIN/STDOUT Supported formats: gzip zopfli xz lzma bzip2 lz4 lz4_legacy lz4_lg decompress <infile> [outfile] Detect format and decompress <infile>, optionally to [outfile] <infile>/[outfile] can be '-' to be STDIN/STDOUT Supported formats: gzip zopfli xz lzma bzip2 lz4 lz4_legacy lz4_lg 1|ASUS_I006D:/ #

The Magisk binary magisk can be used while booted into TWRP if the used TWRP is able to mount the volume for /data; for details see the section Using the magisk binary while the phone is booted into TWRP in the post Some hints for using Magisk on Android phones

If Magisk is installed in the Android OS on the phone and the booted TWRP is able to mount the volume for /data the Magisk binary magisk can also be used in an session in TWRP.
For details see the section Using the magisk binary while the phone is booted into TWRP in the post Some hints for using Magisk on Android phones.

magisk usage

1|ASUS_I006D:/ # /data/adb/magisk/magisk64 Magisk - Multi-purpose Utility Usage: magisk [applet [arguments]...] or: magisk [options]... Options: -c print current binary version -v print running daemon version -V print running daemon version code --list list all available applets --remove-modules remove all modules and reboot --install-module ZIP install a module zip file Advanced Options (Internal APIs): --daemon manually start magisk daemon --stop remove all magisk changes and stop daemon --[init trigger] callback on init triggers. Valid triggers: post-fs-data, service, boot-complete, zygote-restart --unlock-blocks set BLKROSET flag to OFF for all block devices --restorecon restore selinux context on Magisk files --clone-attr SRC DEST clone permission, owner, and selinux context --clone SRC DEST clone SRC to DEST --sqlite SQL exec SQL commands to Magisk database --path print Magisk tmpfs mount path --denylist ARGS denylist config CLI Available applets: su, resetprop 1|ASUS_I006D:/ #

How to install TWRP into the boot partition using CLI commands from within a booted TWRP?

Use the script

install_twrp_from_within_twrp.sh

Download from here: https://bnsmb.de/files/public/Android/install_twrp_from_within_twrp.sh

How to install TWRP after an OS Upgrade from within the running Android OS?

see How to Install TWRP into the boot partition again after the installation of an OS update

How to add additional files to an existing TWRP image?

see How to add additional files to an TWRP image

How to install TWRP into the boot partition via script?

see How to install TWRP via script

How to build a TWRP image?

see How to compile TWRP for the ASUS Zenfone 8

How to install an OS image using the twrp command in a script

see How to install an OS image using the TWRP binary twrp?

How to use TWRP if LineageOS 20.x is installed

As of 26.06.2023 booting the phone from the TWRP recovery image using the command

sudo fastboot boot <twrp_image>

Does not work anymore if LineageOS 20.x is installed on the phone
.
See How to use TWRP if LineageOS 20.x is installed for a workaround

History of this entry

02.12.2022 /bs
initial release

17.12.2022 /bs
added infos about how to use the binary twrp to install an OS image

26.12.2022 /bs
added infos about the logile used by TWRP

23.04.2023 /bs
added info about how to decrypt the data partition

30.04.2023 /bs
added infos about the file used for the TWRP settings

12.03.2023 /bs
added infos about using the twrp executable outside of TWRP

19.05.2023 /bs
added infos about how to check if the data partition is encrypted
updated the infos about the log file used by TWRP
added infos about how to install TWRP from within a running Android OS

26.06.2023 /bs
added the infos about how to use TWRP with LineageOS 20.x

How to enable root access using Magisk in a script

URL: https://xdaforums.com/t/how-to-enable-root-access-using-magisk-in-a-script.4527035/

How to enable root access using Magisk in a script

Magisk uses the table policies in the Magisk database /data/adb/magisk.db to store the list of root enabled apps.

This Howto is about how to change the entries in that table via script.

Caution:

Changing the table via script is necessary for the automatic installation and configuration of the Android OS. Of course you can do a lot of mischief with it and bypass the security checks of Android and Magisk i.e. this should only be used with caution!
On the other hand, who rooted her phone via Magisk should know what she is doing anyway ....

Anyway -- let's start:

Note:

Most of the commands in this HowTo must be done by the user root.

magisk.db is a sqlite database so to list the table sqlite3 can be used, e.g.:

ASUS_I006D:/ # /data/adb/sqlite3 /data/adb/magisk.db 'select * from policies ;' 2000|2|0|1|1 10143|2|0|1|1 10134|2|0|1|1 10141|2|0|1|1 ASUS_I006D:/ #

or in a human readable format:

ASUS_I006D:/ # /data/adb/sqlite3 -table /data/adb/magisk.db 'select * from policies ;' +-------+--------+-------+---------+--------------+ | uid | policy | until | logging | notification | +-------+--------+-------+---------+--------------+ | 2000 | 2      | 0     | 1       | 1            | | 10143 | 2      | 0     | 1       | 1            | | 10134 | 2      | 0     | 1       | 1            | | 10141 | 2      | 0     | 1       | 1            | +-------+--------+-------+---------+--------------+ ASUS_I006D:/ #

Not all sqlite3 binaries support the parameter -table : for sqlite binaries not supporting the parameter -table the parameter -column -header can be used for a human readble output, e.g.:

ASUS_I006D:/ # /data/adb/sqlite3 -column -header /data/adb/magisk.db "select * from policies ;" uid    policy until logging notification ----- ------ ----- ------- ------------ 2000   2       0      1        1           10134 2       0      1        1           10141 2       0      1        1           10143 2       0      1        1           10129 2       0      1        1           10122 2       0      1        1           10133 3       0      1        1           ASUS_I006D:/ #

sqlite3 is not part of the Android OS; it must be installed either by copying the sqlite3 binary to the phone (see also this thread : https://xdaforums.com/t/new-sqlite3-binary-v3-40-0-for-all-devices.4273049/ ) or by installing a Magisk module with sqlite3

Fortunately the binary magisk from Magisk can also work with SQL databases. So the entries of the table policies can also be printed using the magisk binary, e.g:

ASUS_I006D:/ # magisk --sqlite 'select * from policies ;' logging=1|notification=1|policy=2|uid=2000|until=0 logging=1|notification=1|policy=2|uid=10143|until=0 logging=1|notification=1|policy=2|uid=10134|until=0 logging=1|notification=1|policy=2|uid=10141|until=0 ASUS_I006D:/ #

To add another entry to the table policies either sqlite3 or magisk can be used, example:

ASUS_I006D:/ # /data/adb/sqlite3 -table /data/adb/magisk.db 'select * from policies ;' +-------+--------+-------+---------+--------------+ | uid | policy | until | logging | notification | +-------+--------+-------+---------+--------------+ | 2000 | 2      | 0     | 1       | 1            | | 10143 | 2      | 0     | 1       | 1            | | 10134 | 2      | 0     | 1       | 1            | | 10141 | 2      | 0     | 1       | 1            | +-------+--------+-------+---------+--------------+ ASUS_I006D:/ #

-> insert an entry for the UID 10888 in the table policies using magisk :

ASUS_I006D:/ # magisk --sqlite 'INSERT INTO policies (uid,policy,until,logging,notification) values(10888,2,0,1,1);'                                                            ASUS_I006D:/ # ASUS_I006D:/ # /data/adb/sqlite3 -table /data/adb/magisk.db 'select * from policies ;' +-------+--------+-------+---------+--------------+ | uid | policy | until | logging | notification | +-------+--------+-------+---------+--------------+ | 2000 | 2      | 0     | 1       | 1            | | 10143 | 2      | 0     | 1       | 1            | | 10134 | 2      | 0     | 1       | 1            | | 10141 | 2      | 0     | 1       | 1            | | 10888 | 2      | 0     | 1       | 1            | +-------+--------+-------+---------+--------------+ ASUS_I006D:/ #

-> insert an entry for the UID 10999 in the table policies using sqlite3 :

ASUS_I006D:/ # /data/adb/sqlite3 -table /data/adb/magisk.db 'INSERT INTO policies (uid,policy,until,logging,notification) values(10999,2,0,1,1);'                                   ASUS_I006D:/ # ASUS_I006D:/ # /data/adb/sqlite3 -table /data/adb/magisk.db 'select * from policies ;' +-------+--------+-------+---------+--------------+ | uid | policy | until | logging | notification | +-------+--------+-------+---------+--------------+ | 2000 | 2      | 0     | 1       | 1            | | 10143 | 2      | 0     | 1       | 1            | | 10134 | 2      | 0     | 1       | 1            | | 10141 | 2      | 0     | 1       | 1            | | 10888 | 2      | 0     | 1       | 1            | | 10999 | 2      | 0     | 1       | 1            | +-------+--------+-------+---------+--------------+ ASUS_I006D:/ #

To change an existing entry use the SQL command REPLACE, e.g.

# # using the magisk binary to change the entry for the UID 10888 # magisk --sqlite 'REPLACE INTO policies (uid,policy,until,logging,notification) values(10888,2,0,1,1);' # # using the sqlite3 binary to change the entry for the UID 10999 # /data/adb/sqlite3 -table /data/adb/magisk.db 'REPLACE INTO policies (uid,policy,until,logging,notification) values(10999,2,0,1,1);'

To delete an entry use the SQL command DELETE, e.g.:

# # use magisk to delete the entry for the UID 10999 # magisk --sqlite 'DELETE FROM policies where (uid=10999) ; ' # # use sqlite3 to delete the entry for the UID 10888 # /data/adb/sqlite3 -table /data/adb/magisk.db 'DELETE FROM policies where (uid=10888);'

The uid in the table policies is the UID from the packages installed on the phone.

To get the list of installed packages with UIDs use this command:

pm list packages -U

example:

130|ASUS_I006D:/ # pm list packages -U package:org.microg.nlp.backend.nominatim uid:10071 package:com.qualcomm.qti.gpudrivers.lahaina.api30 uid:10079 package:com.android.theme.icon.teardrop uid:10008 .... package:com.qualcomm.qtil.aptxacu uid:10110 package:com.android.pacprocessor uid:10062 package:com.android.messaging uid:10091 package:com.android.theme.icon.square uid:10007 ASUS_I006D:/ #

To get the UID for a specific package use

pm list packages -U "<appname>"

example:

ASUS_I006D:/ # pm list packages -U com.android.shell | cut -f3 -d ":"                                                                                       2000 ASUS_I006D:/ #

The value for policy in the table policies defines the root access status for an app: the value for root enabled is 2. So to list the UIDs of all apps with enabled root access use this command:

ASUS_I006D:/ # magisk --sqlite "SELECT uid FROM policies WHERE policy = '2';" uid=2000 uid=10143 uid=10134 uid=10141 ASUS_I006D:/ #
... or use the handy little script from this thread https://xdaforums.com/t/magisk-general-support-discussion.3432382/page-2681:

130|ASUS_I006D:/ # sh /sdcard/Download/list_magisk_root_permissions.sh Packages with root granted: package:com.android.shell uid:2000 package:com.mixplorer uid:10143 package:com.keramidas.TitaniumBackup uid:10134 package:io.github.muntashirakon.AppManager uid:10141 ASUS_I006D:/ #

Note:

The value for policy in the table policies for apps with disabled root access is 1.

Scripts

To automate enabling root access via changing the entries in the Magisk database I wrote a little script:

enable_root_access_via_magisk.sh

Usage:

ASUS_I006D:/ # /data/adb/enable_root_access_via_magisk.sh -h     enable_root_access_via_magisk.sh [-f] [-H|--help] [id1 ... id#] 1|ASUS_I006D:/ #

Verbose Usage:

enable_root_access_via_magisk.sh --help

ASUS_I006D:/ # /data/adb/enable_root_access_via_magisk.sh --help enable_root_access_via_magisk.sh - script to enable root access by adding entries to the Magisk database Usage: enable_root_access_via_magisk.sh [-f] [-H|--help] [id1 ... id#] "id#" are the UIDs of the apps for which root access should be enabled If the script is running in the Android OS "id#" can also be the app name. The script was tested with these Magisk versions Magisk 24.3 Magisk 25.1 Magisk 25.2 The format of the database entries for other Magisk versions may be different. Therefor to use it for databases created by other Magisk versions use the parameter -f The script can run in the Android OS or in an adb shell while booted from TWRP. The UIDs must be numeric if running in TWRP; to get the UID for an installed app use the command pm list packages -U <app_name> | cut -f3 -d : e.g. pm list packages -U com.android.shell | cut -f3 -d : while the Android OS is running. The script uses the binary sqlite3 if found (either in the PATH or anywhere in /data/adb or /sdcard/Download ). If no sqlite3 binary is found the magisk binary will be used. If the Magisk binary is available in the PATH, this is used, otherwise the Magisk binary /data/adb/magisk/magisk64 (or /data/adb/magisk/magisk32 for 32-bit CPUSs) is used. Environment variables used by the script if set: MAGISK_DATA_DIR (Current value: "/data/adb") BIN_DIR (Current value: "/system/bin") TMPDIR (Current value: "/data/local/tmp") SQLITE3 (Current value: "/system/xbin/sqlite3") 1|ASUS_I006D:/ #

enable_root_access_via_magisk.sh example

# # running enable_root_acces_via_magisk.sh on the phone # ASUS_I006D:/ $ su - -c "sh /cache/enable_root_access_via_magisk.sh com.mixplorer com.keramidas.TitaniumBackup io.github.muntashirakon.AppManager com.matoski.adbm com.fox2code.mmm" Enabling root access for apps by adding entries to the Magisk database Running in the Android OS Using the Magisk binary "/system/bin/magisk" Using the Magisk database "/data/adb/magisk.db" Checking the prerequisites ... Checking if the required executables exist ... OK, the file "/system/bin/magisk" exists and is executable Checking if the required data files exist ... OK, the file "/data/adb/magisk.db" exists Searching the sqlite3 binary in /data/adb/ ... Searching the sqlite3 binary in /sdcard/Download/ ... No sqlite3 executable found - using the binary "/system/bin/magisk" to process the Magisk database Checking if the Magisk daemon is already running ... 25.1:MAGISK:R OK, the Magisk daemon is already running Checking the parameter ... The UID for "com.mixplorer" is "10143" The UID for "com.keramidas.TitaniumBackup" is "10135" The UID for "io.github.muntashirakon.AppManager" is "10142" The UID for "com.matoski.adbm" is "10137" The UID for "com.fox2code.mmm" is "10141" OK, all parameter are okay OK, the installed Magisk version "25.1" is supported Creating a backup of the existing Magisk database in "/data/adb/magisk.db.4929.bkp" ... Reading the current entries from the Magisk database ... Current entries in the Magisk database table "policies": - logging=1|notification=1|policy=1|uid=10137|until=0 logging=1|notification=1|policy=2|uid=2000|until=0 Enabling the root access for these app UIDs: 10143 10135 10142 10137 10141 Processing the UID "10143" ... Enabling root access for the UID "10143" now ... OK, root access successfully enabled for the UID "10143" Processing the UID "10135" ... Enabling root access for the UID "10135" now ... OK, root access successfully enabled for the UID "10135" Processing the UID "10142" ... Enabling root access for the UID "10142" now ... OK, root access successfully enabled for the UID "10142" Processing the UID "10137" ... Enabling root access for the UID "10137" now ... OK, root access successfully enabled for the UID "10137" Processing the UID "10141" ... Enabling root access for the UID "10141" now ... OK, root access successfully enabled for the UID "10141" Summary root access enabled for 5 UID(s): 10143 10135 10142 10137 10141 Current entries in the Magisk database table "policies": - logging=1|notification=1|policy=1|uid=10137|until=0 logging=1|notification=1|policy=2|uid=2000|until=0 logging=1|notification=1|policy=2|uid=10143|until=0 logging=1|notification=1|policy=2|uid=10135|until=0 logging=1|notification=1|policy=2|uid=10142|until=0 logging=1|notification=1|policy=2|uid=10141|until=0 ASUS_I006D:/ $

enable_root_access_via_magisk.sh can run in the Android OS or in a adb shell in a running TWRP. If running in the Android OS the script needs an already enabled root access for the shell so it can not be used that way to configure the initial root access for the shell.

Therefor I wrote another script to automate enabling the root access for the shell after the initial installation:

init_magisk_db.sh

init_magisk_db.sh boots the phone from the TWRP image, copies the script enable_root_access_via_magisk.sh to the phone, and executes enable_root_access_via_magisk.sh then on the phone to enable the initial root access.
This is useful to automatically configure a root enabled shell that then can be used in other scripts to complete the installation and configuration of the phone.

Usage:

[xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ ./init_magisk_db.sh -h init_magisk_db.sh version - v1.0.0.0 - shell script to enable root access via Magisk for specified apps init_magisk_db.sh [-h|help|-H] [--list|-l] [--reboot|--noreboot] [apps=app1,..,app#] [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$

Verbose usage:

init_magisk_db.sh --help

[xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ ./init_magisk_db.sh --help init_magisk_db.sh version - v1.0.0.0 - shell script to enable root access via Magisk for specified apps Usage init_magisk_db.sh [-h|help|-H] [--list|-l] [--force|-f] [--reboot|--noreboot] [apps=app1,..,app#] All parameter are optional. The parameter can be used in any order. Use the parameter "help" or "-H" to print the detailed usage help; use the parameter "-h" to print only the short usage help If the parameter "--list" is used the script will only print all apps for which root access would be enabled and exit Use the parameter "--force" if the used Magisk version is not official supported by the script enable_root_access_via_magisk.sh. If the parameter "--reboot" is used the script will reboot the phone after enabling the root access; to disable the automatic reboot use the parameter "--noreboot". Default is "--noreboot". The parameter "apps=app#" can be used to define the apps for which root access should be enabled. apps can be either the "name" of an app or the UID of an app. e.g. apps=com.android.shell,10123,com.keramidas.TitaniumBackup The value of the parameter "apps=" will overwrite the default list of apps to be root enabled hardcoded in the script; to append the apps to the list of hardcoded apps use a leading "+", e.g. apps="+app1,..,app#". The parameter can be used more then once; the leading "+" is mandatory in this case for all but the first occurance. To change some of the values used by the script you can set environment variables before starting the script: Set the environment variable REBOOT to "yes" before starting the script to automatically reboot the phone after enabling the root access Set the environment variable REBOOT to "no" before starting the script to disable the automatic reboot after enabling the root access See also the source code of the script boot_phone_from_twrp.sh for environment variables supported by this script Notes Use the Android command pm list packages in an adb shell to get the names of the installed apps; use the Android command pm list packages -U <appname> | cut -f3 -d ":" to get the UID of an app. Prerequisites - a phone with installed Magisk; the script was tested with Magisk v25.2 - working adb and fastboot binaries on the PC - the phone must be connected via USB - access via adb must be enabled on the phone Details If the list of apps for enabling root access contains one or more app names the script will first boot the phone into the normal mode if not already running in that mode to get the UIDs for these apps. The helper scripts boot_phone_from_twrp.sh and enable_root_access_via_magisk.sh are required -- see the source code of the script [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ ./boot_phone_from_twrp.sh -h boot_phone_from_twrp.sh v2.0.0.0 - boot a phone from a TWRP image boot_phone_from_twrp.sh [-h|help|-H] [serial=#sn|s=#sn] [wait=n] [force|noforce] [reboot|noreboot] [checkonly] [twrp_image] [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ ./boot_phone_from_twrp.sh --help boot_phone_from_twrp.sh v2.0.0.0 - boot a phone from a TWRP image Usage boot_phone_from_twrp.sh [-h|help|-H] [serial=#sn|s=#sn] [wait=n] [force|noforce] [reboot|noreboot] [checkonly] [twrp_image] All parameter are optional Use the parameter "help" or "-H" to print the detailed usage help; use the parameter "-h" to print only the short usage help Use the parameter "serial=#sn" to define the serialnumber of the phone to process. This parameter is only necessary if there are more then one phones connected via USB and the environment variable SERIAL_NUMBER is not set. The value for the parameter "wait" is the maximum number of seconds to wait after booting from the TWRP image until the adb deamon is ready to use. The default value is 10 seconds. Use the parameter "force" to reboot the phone from the TWRP image even if it's booted from a TWRP installed in the boot or recovery partition. Use the parameter "noforce" to disable rebooting the phone from the TWRP image if it's booted from a TWRP installed in the boot or recovery partition. This is the default setting. Use the parameter "reboot" to reboot the phone from the TWRP image even if it's already booted from an TWRP image. Use the parameter "noreboot" to disable rebooting the phone from the TWRP image if it's already booted from an TWRP image This is the default setting. Use the parameter "checkonly" to only retrieve the current boot mode of the phone; the script will return the current state via return code; the possible values are 1 to 5 or one of the return codes indicating an error. The parameter "twrp_image" can be used to define another TWRP image to use. The parameter is optional - the default for "twrp_image" is the value of the environment variable TWRP_IMAGE; the default for the variable TWRP_IMAGE is hardcoded in the script in the variable DEFAULT_TWRP_IMAGE (= the TWRP image for the ASUS Zenfone 8) Parameter overwrite the values defined via environment variables. Returncodes: 0 - the phone was successfully booted from the TWRP image 1 - the phone is already booted from the TWRP image 2 - the phone is booted from TWRP installed in the boot or recovery partition 3 - the phone is booted into normal mode 4 - the phone is booted into bootloader 5 - the phone is booted into the fastbootd 9 - too many phones connected 10 - usage help printed and exited 100 - invalid parameter found 101 - TWRP image not found 102 - booting the phone into the bootloader failed 103 - booting the phone into the TWRP failed 252 - access to the phone failed 253 - requirement check failed (e.g. one or more required executables not found, etc) 254 - unknown error The phone to boot must be attached via USB. The phone can be either in fastboot mode, in normal mode with enabled adb support, in the boot loader, or already booted from an installed TWRP or an TWRP image To change some of the values used by the script you can set environment variables before starting the script: Set the environment variable TWRP_IMAGE to the name of the TWRP image file that should be used (the parameter of the script will overwrite the variable) Set the environment variable SERIAL_NUMBER to the serial number of the phone to patch if there is more then one phone connected via USB Set the environment variable ADB_OPTIONS to the options to be used with the adb command Set the environment variable FASTBOOT_OPTIONS to the options to be used with the fastboot command Set the environment variable FAST_BOOT_WAIT_TIME to the maximum number of seconds to wait after booting the phone from the bootloader (default are 60 seconds) Set the environment variable ADB_BOOT_WAIT_TIME to the maximum number of seconds to wait after booting the phone from the TWRP image (default are 60 seconds) Set the environment variable ADB to the adb executable that should be used; default: search adb in the PATH Set the environment variable FASTBOOT to the fastboot executable that should be used; default: search fastboot in the PATH Set the environment variable TIMEOUT to the timeout executable that should be used; default: search timeout in the PATH The default TWRP image to use is "/data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D.img" [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ ./init_magisk_db.sh -h init_magisk_db.sh version - - shell script to enable root access via Magisk for specified apps init_magisk_db.sh [-h|help|-H] [--list|-l] [--force|-f] [--reboot|--noreboot] [apps=app1,..,app#] [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$

init_magisk_db.sh example

[xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ ./init_magisk_db.sh apps=com.android.shell init_magisk_db.sh version - - shell script to enable root access via Magisk for specified apps Using the TWRP helper script "./boot_phone_from_twrp.sh" Using the Magisk helper script "./enable_root_access_via_magisk.sh" The apps for which root access should be enabled are: com.android.shell Apps that should be enabled are: com.android.shell UIDs that should be enabled are: Found one more app names in the list of apps Reading the helper script "boot_phone_from_twrp.sh" ... The script is running as user "xtrnaw7" -- will use "sudo" for the fastboot commands ... Will use the attached phone with the serial number "M6AIB760D0939LX" Using the default TWRP image: "/data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D.img" Using the options " -s M6AIB760D0939LX " for the adb commands Using the options " -s M6AIB760D0939LX " for the fastboot commands Checking the script prerequisites ... Will wait up to 60 second(s) after booting the phone into the bootloader Will wait up to 60 second(s) after booting the phone from the TWRP image Will wait up to 10 second(s) until the adb daemon is ready to use Retrieving the current status of the phone ... The phone is currently booted from the Android OS Retrieving the UIDs for the app names ... Retrieving the UID for the app "com.android.shell" ... The UID for the app "com.android.shell" is "2000" Will enable the root access via Magisk for these UIDs: 2000 The phone is booted into the Android OS Booting the phone into the bootloader now ... Waiting up to 60 seconds for the boot into the fastboot mode ... The phone is booted into the bootloader Booting the phone from the TWRP image "/data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D.img" now ... Sending 'boot.img' (98304 KB) OKAY [ 2.809s] Booting OKAY [ 10.360s] Finished. Total time: 13.215s Waiting up to 60 seconds for the boot of the phone .... Waiting up to 10 seconds for the adb daemon to get ready .. ... the adb daemon is ready after second(s) The phone is booted into TWRP: "twrp_I006D" version "3.7.0_12-0" Copying the Magisk helper script "./boot_phone_from_twrp.sh" to on the phone ... ./enable_root_access_via_magisk.sh: 1 file pushed, 0 skipped. 321.7 MB/s (22461 bytes in 0.000s) Executing the Magisk helper script "/cache/enable_root_access_via_magisk.sh" on the phone now ... Enabling root access for apps by adding entries to the Magisk database OK, running in TWRP TWRP is booted from an image file Using the Magisk binary "/data/adb/magisk/magisk64" Using the Magisk database "/data/adb/magisk.db" Checking the prerequisites ... Checking if the required executables exist ... OK, the file "/data/adb/magisk/magisk64" exists and is executable Checking if the required data files exist ... OK, the file "/data/adb/magisk.db" exists Searching the sqlite3 binary in /data/adb/ ... Searching the sqlite3 binary in /sdcard/Download/ ... No sqlite3 executable found - using the binary "/data/adb/magisk/magisk64" to process the Magisk database Checking if the Magisk daemon is already running ... No daemon is currently running! The Magisk daemon is not running -- starting the Magisk Daemon now ... Checking if the Magisk daemon start was successfull ... 25.1:MAGISK:R OK, the Magisk daemon is now running Checking the parameter ... The parameter "2000" is okay OK, all parameter are okay OK, the installed Magisk version "25.1" is supported Creating a backup of the existing Magisk database in "/data/adb/magisk.db.565.bkp" ... Reading the current entries from the Magisk database ... Current entries in the Magisk database table "policies": - logging=1|notification=1|policy=1|uid=10137|until=0 Enabling the root access for these app UIDs: 2000 Processing the UID "2000" ... Enabling root access for the UID "2000" now ... OK, root access successfully enabled for the UID "2000" Summary root access enabled for 1 UID(s): 2000 Current entries in the Magisk database table "policies": - logging=1|notification=1|policy=1|uid=10137|until=0 logging=1|notification=1|policy=2|uid=2000|until=0 root access for the UIDs via Magisk successfully enabled *** Press return to reboot the phone now Rebooting the phone now ... Booting the Android OS using the command "adb reboot " now ... [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ # # Test # [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$ adb shell su - -c id uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 [xtrnaw7@t15g /data/develop/android/scripts_on_linux]$

Notes:

init_magisk_db.sh needs a working adb access to the phone.

If one or more of the values for the parameter apps= for init_magisk_db.sh are app names init_magisk_db.sh will first boot the phone into the Android OS (if not already booted into the Android OS) to retrieve the UIDs for these apps (for this task no root access is necessarry).

init_magisk_db.sh uses the script boot_phone_from_twrp.sh for booting the phone into the TWRP image (the script is also attached to this post)

This method can also be used to give apps root rights that they do not require ... whether this is useful everyone must decide for themselves.

All three scripts mentioned in this post are also available on my home page:

boot_phone_from_twrp.sh

enable_root_access_via_magisk.sh

init_magisk_db.sh

Note that the Magisk App will silently remove all invalid entries from the table policies in the Magisk database; dito for entries for non-existent UIDs .

Where does the Magisk App store the settings?

URL: https://xdaforums.com/t/where-does-the-magisk-app-store-the-settings.4527129/

Where does the Magisk App store the settings?

The Magisk App uses the standard location for storing the settings of the App - on my phone this is the directory

/data/user_de/0/com.topjohnwu.magisk/shared_prefs

e.g.
ASUS_I006D:/ # ls -l /data/user_de/0/com.topjohnwu.magisk/shared_prefs total 8 -rw-rw---- 1 u0_a129 u0_a129 163 2022-12-06 12:20 com.topjohnwu.magisk_preferences.xml -rw-rw---- 1 u0_a129 u0_a129 116 2022-12-04 15:09 su_timeout.xml ASUS_I006D:/ # [/CODE] The files in that directory are standard xml files, e.g. [CODE] ASUS_I006D:/ # cat /data/user_de/0/com.topjohnwu.magisk/shared_prefs/com.topjohnwu.magisk_preferences.xml<?xml version='1.0' encoding='utf-8' standalone='yes' ?> <map> <string name="update_channel">-1</string> <string name="su_auto_response">0</string> </map> ASUS_I006D:/ #

The file com.topjohnwu.magisk_preferences.xml is used to store the settings of the Magisk App.

To change one of the settings for the Magisk App via script just change the value for the corresponding entry in the XML file (or create the entry if it's missing).

For example the entry for the setting "Automatic Response" is "su_auto_response" and the known values for this entry are: Grant = 2, Deny = 1 , Prompt = 0

If the Magisk App is currently running it must be stopped (for example via the Android Settings) to activate the new setting.

To stop the Magisk app via CLI command use

am force-stop com.topjohnwu.magisk

The known values for this file are listed in the source code for the app

https://github.com/topjohnwu/Magisk/blob/master/app/core/src/main/java/com/topjohnwu/magisk/core/Config.kt

As of 06.12.2022 for the Magisk App version 25.2 the defined entries and values for the file are:

Known keys and values for the settings xml file

object Key { // db configs const val ROOT_ACCESS = "root_access" const val SU_MULTIUSER_MODE = "multiuser_mode" const val SU_MNT_NS = "mnt_ns" const val SU_BIOMETRIC = "su_biometric" const val ZYGISK = "zygisk" const val DENYLIST = "denylist" const val SU_MANAGER = "requester" const val KEYSTORE = "keystore" // prefs const val SU_REQUEST_TIMEOUT = "su_request_timeout" const val SU_AUTO_RESPONSE = "su_auto_response" const val SU_NOTIFICATION = "su_notification" const val SU_REAUTH = "su_reauth" const val SU_TAPJACK = "su_tapjack" const val CHECK_UPDATES = "check_update" const val UPDATE_CHANNEL = "update_channel" const val CUSTOM_CHANNEL = "custom_channel" const val LOCALE = "locale" const val DARK_THEME = "dark_theme_extended" const val REPO_ORDER = "repo_order" const val SHOW_SYSTEM_APP = "show_system" const val DOWNLOAD_DIR = "download_dir" const val SAFETY = "safety_notice" const val THEME_ORDINAL = "theme_ordinal" const val BOOT_ID = "boot_id" const val ASKED_HOME = "asked_home" const val DOH = "doh" } object Value { // Update channels const val DEFAULT_CHANNEL = -1 const val STABLE_CHANNEL = 0 const val BETA_CHANNEL = 1 const val CUSTOM_CHANNEL = 2 const val CANARY_CHANNEL = 3 const val DEBUG_CHANNEL = 4 // root access mode const val ROOT_ACCESS_DISABLED = 0 const val ROOT_ACCESS_APPS_ONLY = 1 const val ROOT_ACCESS_ADB_ONLY = 2 const val ROOT_ACCESS_APPS_AND_ADB = 3 // su multiuser const val MULTIUSER_MODE_OWNER_ONLY = 0 const val MULTIUSER_MODE_OWNER_MANAGED = 1 const val MULTIUSER_MODE_USER = 2 // su mnt ns const val NAMESPACE_MODE_GLOBAL = 0 const val NAMESPACE_MODE_REQUESTER = 1 const val NAMESPACE_MODE_ISOLATE = 2 // su notification const val NO_NOTIFICATION = 0 const val NOTIFICATION_TOAST = 1 // su auto response const val SU_PROMPT = 0 const val SU_AUTO_DENY = 1 const val SU_AUTO_ALLOW = 2 // su timeout val TIMEOUT_LIST = intArrayOf(0, -1, 10, 20, 30, 60) // repo order const val ORDER_NAME = 0 const val ORDER_DATE = 1 }

If in doubt what key or value to use just create a backup of the config file; change the approbiate key in the setting in the Magisk App and compare the config file with the just created backup file.

How to install an OS image using the TWRP binary twrp

URL: https://xdaforums.com/t/how-to-install-an-os-image-using-the-twrp-binary-twrp.4531345/

How to install an OS image using the TWRP binary twrp

Update 28.04.2024:

see also the entry How to install an OS image via sideload using the TWRP binary twrp for some new script features not documented in this entry

The most important step to automate the installation and configuration of the Android OS is the installation of an OS image via script without user intervention. One method to do that is using the binary twrp from the recovery TWRP .

The command to install an OS image with twrp is

/system/bin/twrp install /path/to/update.zip

The usage for the binary twrp is (see also here)

twrp usage help

I006D:/ # /system/bin/twrp --help TWRP openrecoveryscript command line tool, TWRP version 3.7.0_12-0 Allows command line usage of TWRP via openrecoveryscript commands. Some common commands include: install /path/to/update.zip backup <SDCRBAEM> [backupname] restore <SDCRBAEM> [backupname] wipe <partition name> format data sideload set <variable> [value] decrypt <password> [USER ID] remountrw fixperms mount <path> unmount <path> listmounts print <value> mkdir <directory> reboot [recovery|poweroff|bootloader|download|edl] See more documentation at https://twrp.me/faq/openrecoveryscript.html I006D:/ #

The binary twrp can also be used to wipe the data from the phone using the parameter wipe <partition_name>.

The command twrp install will not change the active boot slot for the next reboot so this must be done manually after installing the OS image.

This can be done using the binary bootctl which is also part of TWRP.

The usage for the command bootctl is:

bootctl usage help

ASUS_I006D:/ # /system/bin/bootctl -h /system/bin/bootctl - command-line wrapper for the boot HAL. Usage: /system/bin/bootctl COMMAND Commands: hal-info - Show info about boot_control HAL used. get-number-slots - Prints number of slots. get-current-slot - Prints currently running SLOT. mark-boot-successful - Mark current slot as GOOD. get-active-boot-slot - Prints the SLOT to load on next boot. set-active-boot-slot SLOT - On next boot, load and execute SLOT. set-slot-as-unbootable SLOT - Mark SLOT as invalid. is-slot-bootable SLOT - Returns 0 only if SLOT is bootable. is-slot-marked-successful SLOT - Returns 0 only if SLOT is marked GOOD. get-suffix SLOT - Prints suffix for SLOT. set-snapshot-merge-status STAT - Sets whether a snapshot-merge of any dynamic partition is in progress. Valid STAT values are: none, unknown, snapshotted, merging, or cancelled. get-snapshot-merge-status - Prints the current snapshot-merge status. SLOT parameter is the zero-based slot-number. 64|ASUS_I006D:/ #

Afterwards the phone can be rebooted to activate the new installed OS

So the commands for installing an OS image via twrp are

- boot the phone from the TWRP image

- (optional) use twrp to wipe the data

- copy the OS image file via adb push to the phone

- install the OS image file via the twrp binary

- change the next active boot slot via the binary bootctl.

- reboot the phone

I wrote a simple script to just do that :

install_os_via_twrp.sh

The usage of the script is :

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./install_os_via_twrp.sh -h install_os_via_twrp.sh version - v1.0.0.0 - install an OS image via the TWRP command twrp install_os_via_twrp.sh [-h|help|-H] [--reboot|--noreboot] [force] [wipe|wipeall] [wipe_cache] [wipe_data] [wipe_dalvik] [format_data] [format_metadata] [factory_reset] [os_image_file] [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Use the parameter --help or -H to view the detailed usage help

Detailed usage

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./install_os_via_twrp.sh -H install_os_via_twrp.sh version - v1.0.0.0 - install an OS image via the TWRP command twrp Usage install_os_via_twrp.sh [-h|help|-H] [--reboot|--noreboot] [force] [wipe|wipeall] [wipe_cache] [wipe_data] [wipe_dalvik] [format_data] [format_metadata] [factory_reset] [os_image_file] All parameter are optional, except the parameter for the OS image to install "os_image_file". The parameter can be used in any order. Use the parameter "help" or "-H" to print the detailed usage help; use the parameter "-h" to print only the short usage help If the parameter "--reboot" is used the script will reboot the phone after successfully installing the OS image; to disable the automatic reboot use the parameter "--noreboot". Default is to ask the user for confirmation to reboot the phone. Use the parameter "wipe" or "wipeall" to wipe /data, /cache, and Dalvik before installing the OS image. Use one or more of the parameter "wipe_cache", "wipe_data", or "wipe_dalvik" to only wipe some of the partitions used. Use the parameter "format_metadata" to format the meta data partition; this wipe is NOT included in "wipe_all" Use the parameter "format_data" to format the data partition; this wipe is NOT included in "wipe_all" Use the parameter "factory_reset" to do a factory reset before installing the OS image; a factory reset is done by formatting the data and the metadata partitions. Use the parameter "force" to ignore errors while wiping or formatting the data; without this parameter the script will abort if one of the wipe or format commands fails To change some of the values used by the script these environment variables can be set before starting the script: Set the environment variable REBOOT to "yes" before starting the script to automatically reboot the phone after enabling the root access Set the environment variable REBOOT to "no" before starting the script to disable the automatic reboot after enabling the root access Set the environment variable UPLOAD_DIR_ON_THE_PHONE to set the upload directory for the OS image file on the phone (default dir is /tmp; /tmp is mounted on a ramdiks) See also the source code of the script boot_phone_from_twrp.sh for environment variables supported by this script Prerequisites - the phone must be connected via USB - there must be a working connection to the phone using fastboot and adb - a working TWRP image for the phone must exist The script boot_phone_from_twrp.sh is required by this script -- see the source code of the script [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Note:

The script needs the helper script boot_phone_from_twrp.sh

Sample output of the script installing an new version of the same OS

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ REBOOT=yes /data/develop/android/scripts_on_linux/install_os_via_twrp.sh /data/backup/ASUS_ZENFONE8/omnirom/omni-13/omni-13-202212180505-zenfone8-MICROG.zip install_os_via_twrp.sh version - v1.0.0.0 - install an OS image via the TWRP command twrp Using the TWRP helper script "/data/develop/android/scripts_on_linux/boot_phone_from_twrp.sh" The OS image to install is: "/data/backup/ASUS_ZENFONE8/omnirom/omni-13/omni-13-202212180505-zenfone8-MICROG.zip" Reading the helper script "boot_phone_from_twrp.sh" ... The script is running as user "xtrnaw7" -- will use "sudo" for the fastboot commands ... Will use the attached phone with the serial number "M6AIB760D0939LX" Using the default TWRP image: "/data/backup/ASUS_ZENFONE8/twrp/current_twrp.img" Using the options " -s M6AIB760D0939LX " for the adb commands Using the options " -s M6AIB760D0939LX " for the fastboot commands Checking the script prerequisites ... Will wait up to 60 second(s) after booting the phone into the bootloader Will wait up to 60 second(s) after booting the phone from the TWRP image Will wait up to 10 second(s) until the adb daemon is ready to use Retrieving the current status of the phone ... The phone is currently booted into the Android OS The phone is booted into the Android OS Booting the phone into the bootloader now ... Waiting up to 60 seconds for the boot into the fastboot mode ... The phone is booted into the bootloader Booting the phone from the TWRP image "/data/backup/ASUS_ZENFONE8/twrp/current_twrp.img" now ... Sending 'boot.img' (98304 KB) OKAY [ 2.833s] Booting OKAY [ 10.359s] Finished. Total time: 13.243s Waiting up to 60 seconds for the boot of the phone .... Waiting up to 10 seconds for the adb daemon to get ready .. ... the adb daemon is ready after second(s) The phone is booted into TWRP: "twrp_I006D" version "3.7.0_12-0" Waiting up to 10 seconds for the adb daemon to get ready ... the adb daemon is ready after second(s) Coyping the OS image file "/data/backup/ASUS_ZENFONE8/omnirom/omni-13/omni-13-202212180505-zenfone8-MICROG.zip" to "/tmp/omni-13-202212180505-zenfone8-MICROG.zip" on the phone ... adb: error: failed to read copy response8-MICROG.zip /data/backup/ASUS_ZENFONE8/omnirom/omni-13/omni-13-202212180505-zenfone8-MICROG.zip: 1 file pushed, 0 skipped. 201.8 MB/s (1261116637 bytes in 5.959s) adb: error: failed to read copy response adb: device 'M6AIB760D0939LX' not found Copying the image failed -- we will do another try in 5 seconds ... /data/backup/ASUS_ZENFONE8/omnirom/omni-13/omni-13-202212180505-zenfone8-MICROG.zip: 1 file pushed, 0 skipped. 32.5 MB/s (1261116637 bytes in 37.056s) Installing the OS image "/tmp/omni-13-202212180505-zenfone8-MICROG.zip" into the slot "_b" ... Installing zip file '/tmp/omni-13-202212180505-zenfone8-MICROG.zip' Flashing A/B zip to inactive slot: B Step 1/2Step 2/2Updating partition details... ...done Devices on super may not mount until rebooting recovery. To flash additional zips, please reboot recovery to switch to the updated slot. Done processing script file OS image file "/tmp/omni-13-202212180505-zenfone8-MICROG.zip" successfully installed. Now changing the next active slot to _b ... ... successfully changed the active slot for the next reboot Rebooting the phone now ... Booting the Android OS using the command "adb reboot " now ... [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Sample output of the script installing another OS in the inactive slot

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ /data/develop/android/scripts_on_linux/install_os_via_twrp.sh factory_reset /data/backup/ASUS_ZENFONE8/Lineage-19_Android12/lineage-19.1-20221216-nightly-sake-signed.zip install_os_via_twrp.sh version - v1.0.0.0 - install an OS image via the TWRP command twrp Using the TWRP helper script "/data/develop/android/scripts_on_linux/boot_phone_from_twrp.sh" The OS image to install is: "/data/backup/ASUS_ZENFONE8/Lineage-19_Android12/lineage-19.1-20221216-nightly-sake-signed.zip" Reading the helper script "boot_phone_from_twrp.sh" ... The script is running as user "xtrnaw7" -- will use "sudo" for the fastboot commands ... Will use the attached phone with the serial number "M6AIB760D0939LX" Using the default TWRP image: "/data/backup/ASUS_ZENFONE8/twrp/current_twrp.img" Using the options " -s M6AIB760D0939LX " for the adb commands Using the options " -s M6AIB760D0939LX " for the fastboot commands Checking the script prerequisites ... Will wait up to 60 second(s) after booting the phone into the bootloader Will wait up to 60 second(s) after booting the phone from the TWRP image Will wait up to 10 second(s) until the adb daemon is ready to use Retrieving the current status of the phone ... The phone is currently booted into the Android OS The phone is booted into the Android OS Booting the phone into the bootloader now ... Waiting up to 60 seconds for the boot into the fastboot mode ... The phone is booted into the bootloader Booting the phone from the TWRP image "/data/backup/ASUS_ZENFONE8/twrp/current_twrp.img" now ... Sending 'boot.img' (98304 KB) OKAY [ 2.840s] Booting OKAY [ 10.406s] Finished. Total time: 13.286s Waiting up to 60 seconds for the boot of the phone .... Waiting up to 10 seconds for the adb daemon to get ready .. ... the adb daemon is ready after second(s) The phone is booted into TWRP: "twrp_I006D" version "3.7.0_12-0" Waiting up to 10 seconds for the adb daemon to get ready ... the adb daemon is ready after second(s) Wiping all data including media ... Done. You may need to reboot recovery to be able to use /data again. TWRP will not recreate /data/media on an FBE device. Please reboot into your rom to create /data/media Formatting Metadata using mke2fs... Done. Done processing script file Wiping the metadata ... ++ /usr/bin/adb -s M6AIB760D0939LX shell umount /dev/block/by-name/metadata umount: /dev/block/sda22: Invalid argument ++ /usr/bin/adb -s M6AIB760D0939LX shell mke2fs -F -t ext4 /dev/block/by-name/metadata mke2fs 1.45.4 (23-Sep-2019) Discarding device blocks: done Creating filesystem with 4096 4k blocks and 4096 inodes Allocating group tables: done Writing inode tables: done Creating journal (1024 blocks): done Writing superblocks and filesystem accounting information: done Coyping the OS image file "/data/backup/ASUS_ZENFONE8/Lineage-19_Android12/lineage-19.1-20221216-nightly-sake-signed.zip" to "/tmp/lineage-19.1-20221216-nightly-sake-signed.zip" on the phone ... adb: error: failed to read copy responsey-sake-signed.zip /data/backup/ASUS_ZENFONE8/Lineage-19_Android12/lineage-19.1-20221216-nightly-sake-signed.zip: 1 file pushed, 0 skipped. 313.7 MB/s (1114300349 bytes in 3.387s) adb: error: failed to read copy response adb: device 'M6AIB760D0939LX' not found Copying the image failed -- we will do another try in 5 seconds ... /data/backup/ASUS_ZENFONE8/Lineage-19_Android12/lineage-19.1-20221216-nightly-sake-signed.zip: 1 file pushed, 0 skipped. 32.5 MB/s (1114300349 bytes in 32.719s) Installing the OS image "/tmp/lineage-19.1-20221216-nightly-sake-signed.zip" into the slot "_a" ... Installing zip file '/tmp/lineage-19.1-20221216-nightly-sake-signed.zip' Flashing A/B zip to inactive slot: A Step 1/2Step 2/2Updating partition details... ...done Devices on super may not mount until rebooting recovery. To flash additional zips, please reboot recovery to switch to the updated slot. Done processing script file OS image file "/tmp/lineage-19.1-20221216-nightly-sake-signed.zip" successfully installed. Now changing the next active slot to _a ... ... successfully changed the active slot for the next reboot *** Press return to reboot the phone now Rebooting the phone now ... Booting the Android OS using the command "adb reboot " now ... [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Testenvironment

I tested the script install_os_via_twrp.sh using this hardware and software:

Hardware:

ASUS Zenfone 8

TWRP

TWRP 3.7.0.12

OS Images installed via script

OmniROM 12
OmniROM 13
LineageOS 19

The script does not work for the images with the original Android from ASUS; the OS installation works but the phone does not boot from the new installation (I currently do not know why it does not work...).

The script install_os_via_twrp.sh is also available on my home page:

install_os_via_twrp.sh

Trouble Shooting

To update the installed OS to a new version in most cases neither a wipe nor a factory reset is necessary.

To install a different distribution on the phone in most cases a factory reset is required. (e.g installing LIneageOS if the current OS is OmniROM)

How to install an OS image via sideload using the TWRP binary twrp

URL: https://xdaforums.com/t/how-to-install-an-os-image-via-sideload-using-the-twrp-binary-twrp.4669384/

How to install an OS image via sideload using the TWRP binary twrp

In the entry How to install an OS image using the TWRP binary twrp I describe how to install an OS image in a script using the twrp binary from TWRP with the parameter install.

But the automatic Installation of OS images using the binary twrp from TWRP with the parameter install does not work for all ROM images, e.g. it's not possible to install the Original Android OS from ASUS, LineageOS, or StatiXOS on the ASUS Zenfone 8 using this method.

The sideload function of the twrp binary should therefore be used to install one of these OS images..
.

Example:

To Install the Android OS from ASUS boot the phone from the TWRP image and then issue on the PC:

# optional: retrieve the current active slot # (1 = _b, 0 = -a) # adb shell bootctl get-current-slot # optional: do a factory reset (this is not necessary for OS updates) #
adb shell twrp format data # start the sideload mode of TWRP adb shell twrp sideload # sideload the OS image that should be installed # adb sideload /data/backup/ASUS_ZENFONE8/ASUS_firmware/Android_13/UL-ASUS_I006D-ASUS-33.0210.0210.296-1.1.26-2305-user.zip # wait a few seconds until the sideload mode finished on the phone # change the active boot slot to activate the new installed OS # (1 = b, 0 = a; _b and _a also works) # adb shell bootctl set-active-boot-slot 1

Install the ASUS Android OS via sideload

[ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ adb shell twrp format data Done. You may need to reboot recovery to be able to use /data again. Formatting Metadata using mke2fs... Done. Done processing script file [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ adb shell twrp sideload [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ time adb sideload /data/backup/ASUS_ZENFONE8/ASUS_firmware/Android_13/UL-ASUS_I006D-ASUS-33.0210.0210.296-1.1.26-2305-user.zip * daemon not running; starting now at tcp:5037 * daemon started successfully Total xfer: 1.00x real 6m12.216s user 0m0.004s sys 0m0.006s [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ adb shell bootctl set-active-boot-slot 1 [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $

Notes:

Before starting the sideload installation you may need to perform a wipe or factory reset - depending on which OS is already installed on the phone. This can also be done via the twrp binary using a script.

I added this functionality now to my script to install the Android OS via TWRP: install_os_via_twrp.sh

There are three new parameter for the script:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./install_os_via_twrp.sh -h install_os_via_twrp.sh version - v2.0.0.0 - install an OS image via the TWRP command twrp install_os_via_twrp.sh [-h|help|-H] [--reboot|--noreboot] [force] [wipe|wipeall] [wipe_cache] [wipe_data] [wipe_dalvik] [wipe_all] [format_data] [format_metadata] [factory_reset] [sideload] [nosideload|install] [auto] [os_image_file] [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

The script parameter sideload forces the script to install the operating system image with "twrp sideload"; if the script parameter auto is used, the script decides which method to use for the installation depending on the name of the image that should be installed.
The script parameter nosideload or install forces the script to install the image using "twrp install" - this is also the default method if neither the parameter sideload nor the parameter auto is used.

Detailed usage for the script:

./install_os_via_twrp.sh --help

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./install_os_via_twrp.sh --help install_os_via_twrp.sh version - v2.0.0.0 - install an OS image via the TWRP command twrp Usage install_os_via_twrp.sh [-h|help|-H] [--reboot|--noreboot] [force] [wipe|wipeall] [wipe_cache] [wipe_data] [wipe_dalvik] [wipe_all] [format_data] [format_metadata] [factory_reset] [sideload] [nosideload|install] [auto] [os_image_file] All parameter are optional, except the parameter for the OS image to install "os_image_file". The parameter can be used in any order. Use the parameter "help" or "-H" to print the detailed usage help; use the parameter "-h" to print only the short usage help If the parameter "--reboot" is used the script will reboot the phone after successfully installing the OS image; to disable the automatic reboot use the parameter "--noreboot". Default is to ask the user for confirmation to reboot the phone. Use the parameter "wipe" or "wipeall" to wipe /data, /cache, and Dalvik before installing the OS image. Use one or more of the parameter "wipe_cache", "wipe_data", or "wipe_dalvik" to only wipe some of the partitions used. Use the parameter "format_metadata" to format the meta data partition; this is NOT included in "wipe_all" Use the parameter "format_data" to format the data partition; this is NOT included in "wipe_all" Use the parameter "factory_reset" to do a factory reset before installing the OS image; a factory reset is done by formatting the data and the metadata partitions. Use the parameter "force" to ignore errors while wiping or formatting the data; without this parameter the script will abort if one of the wipe or format commands fails Use the parameter "sideload" to install the image via TWRP sideload feature (this is necessary to install the Original ASUS ROM or the LineageOS) Use the parameter "nosideload" or "install" to force the usage of "twrp install" for installing the image Use the parameter "auto" to automatically select the installation method (install or sideload) depending on the name of the OS image file Using this parameter the script uses "install" for OmniROM images, "sideload" for LineageOS and ASUS Android images, and "sideload" for all unknown OS images Without the parameter "auto" and "sideload" the script uses the TRWP install functionality to install the OS To change some of the values used by the script these environment variables can be set before starting the script: Set the environment variable REBOOT to "yes" before starting the script to automatically reboot the phone after enabling the root access Set the environment variable REBOOT to "no" before starting the script to disable the automatic reboot after enabling the root access Set the environment variable UPLOAD_DIR_ON_THE_PHONE to set the upload directory for the OS image file on the phone (default dir is /tmp; /tmp is mounted on a ramdiks) See also the source code of the script boot_phone_from_twrp.sh for environment variables supported by this script Prerequisites - the phone must be connected via USB - there must be a working connection to the phone using fastboot or adb - a working TWRP image for the phone must exist The script boot_phone_from_twrp.sh is required by this script -- see the source code of the script [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Examples for the script:

Sample output of updating the LineageOS 20 to LineageOS 21 w/o factory reset

# for LineageOS 20 a "special" TWRP is necessary # [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ export TWRP_IMAGE=/data/backup/ASUS_ZENFONE8/Lineage-20/twrp_3.7.0_12-0-I006D_for_lineageOS20-20240423-sake.img [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ time ./install_os_via_twrp.sh sideload wipe_dalvik wipe_cache reboot /data/backup/ASUS_ZENFONE8/Lineage-21/lineage-21.0-20240220-UNOFFICIAL-KSU-GPU_UV-sake.zip install_os_via_twrp.sh version - v2.0.0.0 - install an OS image via the TWRP command twrp Checking the contents of the file "/data/backup/ASUS_ZENFONE8/Lineage-21/lineage-21.0-20240220-UNOFFICIAL-KSU-GPU_UV-sake.zip" ... Using the TWRP helper script "./boot_phone_from_twrp.sh" The OS image to install is: "/data/backup/ASUS_ZENFONE8/Lineage-21/lineage-21.0-20240220-UNOFFICIAL-KSU-GPU_UV-sake.zip" Reading the helper script "boot_phone_from_twrp.sh" ... The script is running as user "xtrnaw7" -- will use "sudo" for the fastboot commands ... Will use the attached phone with the serial number "M6AIB760D0939LX" Using the TWRP image found in the parameter: "/data/backup/ASUS_ZENFONE8/Lineage-20/twrp_3.7.0_12-0-I006D_for_lineageOS20-20240423-sake.img" Using the options "-d -s M6AIB760D0939LX " for the adb commands Using the options " -s M6AIB760D0939LX " for the fastboot commands Checking the script prerequisites ... Will wait up to 60 second(s) after booting the phone into the bootloader Will wait up to 120 second(s) after booting the phone from the TWRP image Will wait up to 30 second(s) until the adb daemon is ready to use Retrieving the current status of the phone ... The phone is currently booted into the Android OS The boot slot is _b. The phone is booted into the Android OS Booting the phone into the bootloader now ... Waiting up to 60 seconds for the boot into the fastboot mode ... The phone is booted into the bootloader Booting the phone from the TWRP image "/data/backup/ASUS_ZENFONE8/Lineage-20/twrp_3.7.0_12-0-I006D_for_lineageOS20-20240423-sake.img" now ... Sending 'boot.img' (98304 KB) OKAY [ 2.827s] Booting OKAY [ 10.359s] Finished. Total time: 13.226s Waiting up to 120 seconds for a working adb connection ... Waiting up to 30 seconds for the adb daemon to get ready ... the adb daemon is ready after 0 second(s) Waiting up to 20 seconds until /data is mounted ... /data is mounted after 0 second(s). Testing if the data partition on the phone is encrypted ... The test directory for the encryption test is "/data/media/0/Download" The data partition on the phone is either not mounted or not encrypted The phone is booted into TWRP version "3.7.0_12-0" Waiting up to 30 seconds for the adb daemon to get ready ... the adb daemon is ready after 0 second(s) Wiping the cache ... Wiping /data/cache/ Done processing script file Wiping the dalvik ... Wiping Dalvik Directories... Cleaned: /data/dalvik-cache... -- Dalvik Directory Wipe Complete! Done processing script file Wiping the dalvik sucesfully done. Waiting up to 10 seconds for a working adb connection Installing the OS image file "/data/backup/ASUS_ZENFONE8/Lineage-21/lineage-21.0-20240220-UNOFFICIAL-KSU-GPU_UV-sake.zip" via sideload option from within TWRP on the phone ... Starting sideload on the phone .. + /usr/bin/adb -d -s M6AIB760D0939LX shell twrp sideload The RC is 0 [2024.04.28 18:35:38] Waiting now 5 seconds ... ..... The phone is booted into the adb sideload mode Now sideloading the image file "/data/backup/ASUS_ZENFONE8/Lineage-21/lineage-21.0-20240220-UNOFFICIAL-KSU-GPU_UV-sake.zip" ... + /usr/bin/adb -d -s M6AIB760D0939LX sideload /data/backup/ASUS_ZENFONE8/Lineage-21/lineage-21.0-20240220-UNOFFICIAL-KSU-GPU_UV-sake.zip Total xfer: 1.00x The RC is 0 Installing the image via "twrp sideload" took 154 seconds(s) [2024.04.28 18:38:17] Waiting now 15 seconds ... ............... Now changing the next active slot to _a ... + /usr/bin/adb -d -s M6AIB760D0939LX shell bootctl set-active-boot-slot 0 The RC is 0 ... successfully changed the active slot for the next reboot Rebooting the phone now ... Parameter for "reboot_phone" are: --nowait The current phone status is "8" Will boot the phone using "adb" Rebooting the phone using the command " /usr/bin/adb -d -s M6AIB760D0939LX reboot " now ... real 3m42.760s user 0m0.151s sys 0m0.821s [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Sample output doing a factory reset and installing the LineageOS 20 on a phone running the ASUS Android 13

In this example we use the script parameter auto so that the script decides which installation method to use.

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ time ./install_os_via_twrp.sh auto factory_reset reboot /data/backup/ASUS_ZENFONE8/Lineage-20/lineage-20.0-20240423-nightly-sake-signed.zip install_os_via_twrp.sh version - v2.0.0.0 - install an OS image via the TWRP command twrp Checking the contents of the file "/data/backup/ASUS_ZENFONE8/Lineage-20/lineage-20.0-20240423-nightly-sake-signed.zip" ... Selecting the installation method depending of the name of the image file ... This seems to be a LineageOS installation image -- using sideload for the installation Using the TWRP helper script "./boot_phone_from_twrp.sh" The OS image to install is: "/data/backup/ASUS_ZENFONE8/Lineage-20/lineage-20.0-20240423-nightly-sake-signed.zip" Reading the helper script "boot_phone_from_twrp.sh" ... The script is running as user "xtrnaw7" -- will use "sudo" for the fastboot commands ... Will use the attached phone with the serial number "M6AIB760D0939LX" Using the default TWRP image: "/data/backup/ASUS_ZENFONE8/twrp/current_twrp.img" Using the options "-d -s M6AIB760D0939LX " for the adb commands Using the options " -s M6AIB760D0939LX " for the fastboot commands Checking the script prerequisites ... Will wait up to 60 second(s) after booting the phone into the bootloader Will wait up to 120 second(s) after booting the phone from the TWRP image Will wait up to 30 second(s) until the adb daemon is ready to use Retrieving the current status of the phone ... The phone is currently booted into the Android OS The boot slot is _a. The phone is booted into the Android OS Booting the phone into the bootloader now ... Waiting up to 60 seconds for the boot into the fastboot mode ... The phone is booted into the bootloader Booting the phone from the TWRP image "/data/backup/ASUS_ZENFONE8/twrp/current_twrp.img" now ... Sending 'boot.img' (98304 KB) OKAY [ 2.808s] Booting OKAY [ 10.390s] Finished. Total time: 13.240s Waiting up to 120 seconds for a working adb connection ..... Waiting up to 30 seconds for the adb daemon to get ready ... the adb daemon is ready after 0 second(s) Waiting up to 20 seconds until /data is mounted ... /data is mounted after 0 second(s). Testing if the data partition on the phone is encrypted ... The test directory for the encryption test is "/data/media/0/Download" The data partition on the phone is either not mounted or not encrypted The phone is booted into TWRP version "3.7.0_12-1" Waiting up to 30 seconds for the adb daemon to get ready ... the adb daemon is ready after 0 second(s) /data is mounted to "/dev/block/dm-5" -- will umount it now ... Unmounted '/data' Done processing script file Umounting /data sucesfully done. Formating /data ... real 0m1.494s user 0m0.001s sys 0m0.003s Done. You may need to reboot recovery to be able to use /data again. TWRP will not recreate /data/media on an FBE device. Please reboot into your rom to create /data/media Formatting Metadata using mke2fs... Done. Done processing script file Formating the data partition sucesfully done. Formating the metadata partition ... ++ /usr/bin/adb -d -s M6AIB760D0939LX shell umount /dev/block/by-name/metadata umount: /dev/block/sda22: Invalid argument ++ /usr/bin/adb -d -s M6AIB760D0939LX shell mke2fs -F -t ext4 /dev/block/by-name/metadata mke2fs 1.45.4 (23-Sep-2019) Discarding device blocks: done Creating filesystem with 4096 4k blocks and 4096 inodes Allocating group tables: done Writing inode tables: done Creating journal (1024 blocks): done Writing superblocks and filesystem accounting information: done Formating the metadata partition sucesfully done. Waiting up to 10 seconds for a working adb connection Installing the OS image file "/data/backup/ASUS_ZENFONE8/Lineage-20/lineage-20.0-20240423-nightly-sake-signed.zip" via sideload option from within TWRP on the phone ... Starting sideload on the phone .. + /usr/bin/adb -d -s M6AIB760D0939LX shell twrp sideload The RC is 0 [2024.04.28 18:24:16] Waiting now 5 seconds ... ..... The phone is booted into the adb sideload mode Now sideloading the image file "/data/backup/ASUS_ZENFONE8/Lineage-20/lineage-20.0-20240423-nightly-sake-signed.zip" ... + /usr/bin/adb -d -s M6AIB760D0939LX sideload /data/backup/ASUS_ZENFONE8/Lineage-20/lineage-20.0-20240423-nightly-sake-signed.zip Total xfer: 1.00x The RC is 0 Installing the image via "twrp sideload" took 156 seconds(s) [2024.04.28 18:26:57] Waiting now 15 seconds ... ............... Now changing the next active slot to _b ... + /usr/bin/adb -d -s M6AIB760D0939LX shell bootctl set-active-boot-slot 1 The RC is 0 ... successfully changed the active slot for the next reboot Rebooting the phone now ... Parameter for "reboot_phone" are: --nowait The current phone status is "8" Will boot the phone using "adb" Rebooting the phone using the command " /usr/bin/adb -d -s M6AIB760D0939LX reboot " now ... real 3m50.116s user 0m0.165s sys 0m0.846s [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Test Environment

I succesfully tested the installation of the Android OS via twrp sideload on the ASUS Zenfone 8 with these ROMs:

Original Android 13 for the ASUS Zenfone 8
LineageOS 19
LineageOS 20
LineageOS 21 (in-official image from https://github.com/mikooomich/android_device_asus_sake/releases)
StatiXOS 14 (7.x) (https://xdaforums.com/t/rom-upsidedowncake-sake-14-statixos-v7-5.4500497/)
OmniROM 13 (this OS can also be installed using twrp install)
OmniROM 14 (this OS can also be installed using twrp install)

Trouble Shooting

The binary bootctl is not included in the newest TWRP versions anymore -- see How to compile the image for TWRP 3.7.0.12-1 w/ the bootctl binary for how to fix that.

Please note that the official TWRP image for the ASUS Zenfone 8 can be not be used to boot the phone if the LineageOS 20 or newer or the current version of the StatiXOS is installed on the phone. See the entry How to use TWRP if LineageOS_20.x is installed for how to create a TWRP image for the LineageOS 20 or newer (the instructions are also valid to create a working TWRP image for the current version of the StatiXOS).

Downgrading the installed operating system (e.g. from LineageOS 21 to ASUS Android 13) with this method may or may not work. If it does not work, install the raw image for the target OS version before installing the new OS image.

Update 26.05.2024

~~For unkown reason the command "adb sideload" must sometimes be executed by the user~~ ~~root~~ ~~in my environment - I don't know why yet. In case the command fails in your environment also with the error message "insufficient permissions for device" just kill the adb daemon and execute~~ ~~adb sideload~~ ~~as user~~ ~~root~~.
With correct udev rules adb sideload also works if the adb was started by a non-root user (see Some hints about adb and fastboot usage in Linux for details)

Download

The script can be downloaded from my homepage:

install_os_via_twrp.sh

The script needs the helper script boot_phone_from_twrp.sh
(see also boot_phone_from_twrp.sh)

History of this entry

28.04.2024
initial release

26.05.2024
corrected the infos about using adb sideload as non-root user

How to "fix" the error "Cannot load Android system. Your data may be corrupt."

URL: https://xdaforums.com/t/how-to-fix-the-error-cannot-load-android-system-your-data-may-be-corrupt.4534237/

How to fix the error Cannot load Android system. Your data may be corrupt.

If a serious error occurs when booting Android, Android boots the phone from the recovery installed on the phone. The standard Android recovery then aborts the boot process with an error message similar to this one:

Cannot load Android system. Your data may be corrupt.

and the only options to continue are "Try again" or "Factory data reset".

The error message is not really useful and most documentation on the Internet that I found recommends a factory reset to fix the problem. In most cases, factory reset works, but it has the disadvantage that you lose all the settings and data on the phone. So in most cases this is the worst option.
However, if the error is not caused by an invalid or missing file in the partitions for /data or /metadata, resetting to factory defaults will not fix the problem, but the data from the phone will still be gone.

Therefor I wrote this little HowTo.

This HowTo is not about how to fix that error -- it only contains some hints to find the reason for the error.

To get more information about the error and also the possibility to backup the phone's data, TWRP can be used:

Just poweroff the phone, reboot the phone into the bootloader, and then reboot the phone from the TWRP image. E.g.

sudo fastboot boot /data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D-enhanced.img

Note:

In most cases, there is no option to turn off the phone in this dialog, so the phone must be turned off using the phone keys.

The key combination to turn off the phone during this dialog depends on the phone; on ASUS Zenfone 8, this key combination must be used:

Press Volume UP + Volume DOWN + Power for 20 or more seconds

After the phone booted into the TWRP image connect via adb to the phone and check the TWRP logfile for a more detailed error message. The logile used by TWRP is /data/recovery/log.gz. Search for the string "Android Rescue Party trigger" in the TWRP logfile.

Note:

/data/recovery/log.gz is a compressed file ; use gzip to uncompress it.

Example:

ASUS_I006D:/ # gzip -cd /data/recovery/log.gz | more Starting TWRP 3.7.0_12-0-d07fdb3c on Sat Jan 10 00:53:15 1970 (pid 403) I:Lun file '/config/usb_gadget/g1/functions/mass_storage.0/lun.0/file' PRODUCT_USE_DYNAMIC_PARTITIONS := true TW_INCLUDE_CRYPTO := true I:TW_BRIGHTNESS_PATH := /proc/lcd_brightness I:Found brightness file at '/proc/lcd_brightness' I:TWFunc::Set_Brightness: Setting brightness control to 420 I:TW_EXCLUDE_ENCRYPTED_BACKUPS := true I:LANG: en I:AB_OTA_UPDATER := true Starting the UI... Skipping adf graphics -- not present in build tree setting DRM_FORMAT_XBGR8888 and GGL_PIXEL_FORMAT_RGBA_8888 setting DRM_FORMAT_XBGR8888 and GGL_PIXEL_FORMAT_RGBA_8888 .... I:Switching packages (TWRP) boot command: boot-recovery I:Startup Commands: Android Rescue Party trigger! Possible solutions? Either: 1. Wipe caches, and/or 2. Format data, and/or 3. Clean-flash your ROM. The reported problem is: '--reason=set_policy_failed:/data/misc' '--reason=set_policy_failed:/data/misc'ro.boot.bootreason=shutdown ro.boot.id.rf=1

If the error message is still not helpful in finding the reason for the error, try a Google search.

And if a factory reset is necessary, you can at least back up your data on the phone before performing the reset.

More infos about Android Rescue Party Trigger

The Android Rescue Party Trigger is an error handling process from Android (see here for more details about this Android functionality).

Android Rescue Party Trigger can also be triggered by Android while the OS is running :

In this case there should be some additional messages in the logfile /data/system/uiderrors.txt.

In addition, and if Magisk is installed, it might be useful to write the logcat messages to a file using the script from this post

https://gist.github.com/niikoo/3f6bd13a69f2d68f3dd51cc667e79bdc

File: /data/adb/post-fs-data.d/0001logcatboot

#!/system/bin/sh mkdir -p /cache/logs /system/bin/logcat -r 1024 -n 9 -v threadtime -f /cache/logs/log >info.log 2>err.log & The script can also be "installed" after the error already occured, to do this:

Boot the phone from the TWRP image; copy the script to the directory /data/adb/post-fs-data.d; make it executable and reboot the phone again from the installed OS to trigger the error again.

Note that logcat is not yet running if the error occurs in the early boot phase.

Caution:

If possible, TWRP will mount /data into the partition used for /data in the Android OS. Therefore, you can view the recovery boot logs in /data/recovery even after restarting the Android operating system as the user root.
However, if mounting the partition for /data in TWRP does not work, /data will be a directory in the root file system, e.g.:

ASUS_I006D:/ # df -h /data Filesystem Size Used Avail Use% Mounted onrootfs 3.2G 101M 3.1G 4% / ASUS_I006D:/ #
In this case the contents of /data/reocvery are lost after rebooting the phone!

Notes:

Update 14.07.2024

Android uses the device /dev/block/by-name/misc to store commands for the bootloader (e.g. to force a reboot into the recovery or to do a factory reset). To avoid endless loops every recovery for an Android phone must clean the device /dev/block/by-name/misc. Therefor the error message is only visible in the logs of the first boot of TWRP.

The active log file for TWRP is /tmp/recovery.log; you can also use that file for the checks. Note that /tmp is mounted on a ramdisk.

On the page https://source.android.com/docs/core/tests/debug/rescue-party are instructions to force an Android Rescue Partry trigger by setting some properties but I did not get that to work neither in the OmniROM 13 nor in the ASUS Android 12.

To avoid the reboot from the TWRP image, install the TWRP recovery on the phone. In this case, Android will automatically boot into the installed TWRP recovery when a fatal error occurs.

How to enable access via adb on a new installed OS

URL: https://xdaforums.com/t/how-to-enable-access-via-adb-on-a-new-installed-os.4535165/

How to enable access via adb on a new installed OS

For automatic installation and configuration of a phone running the Android OS the access via adb is required. "Unfortunately" the access via adb is disabled in most Android distributions (in reality it is of course good that the access via adb is disabled in the default setting! See also the notes regarding the LineageOS at the end of this post)

There are (at least) two ways to enable access via adb:

Manually via the GUI of Android on the phone or via the Android command settings executed by the user root in a shell ,e.g.:

settings put global development_settings_enabled 1 settings put global adb_enabled 1

The latter can also be done in a script in an adb shell -- but of course we need a working adb connection ....

One way around this problem is to use Magisk:

In principle this could be done using an init script for Magisk:

Magisk allows to run scripts while booting the phone (see How to run a script at every boot using Magisk); this feature could be used to enable adb via script.

Unfortunately, this method no longer works in the latest Android versions (Android 12 and Android 13; Android 11 not tested):

Android now uses encryption for most of the files in the subdirectories in /data and to initiate the encryption keys for the files in these directories, the phone must boot once from the installed OS.
To use a Magisk init script to enable the access via adb, we need to create files in the directories /data/adb and /data/misc/adb. If you create these files while booting into TWRP recovery without a configured Android operating system, these files will be created unencrypted, making it impossible to boot the installed Android operating system:

If the Android operating system finds unencrpyted files in these directories at the first reboot, it boots into the installed recovery with the famous error message "Cannot load Android system. Your data may be corrupt" (see How to fix the error "Cannot load Android system. Your data may be corrupt" for details)

Fortunately, the Android operating system does not encrypt files in the directory /data/recovery (and therefore continues booting if there are already files in that directory) and the part of Magisk in the boot partition does not need the Magisk files /data/adb.

So to enable the access via adb using Magisk, we can create a script in the directory /data/recovery and a new Init .rc file for the Android operating system to execute that script while the Android OS is booting - for details about how to do that see How to trigger an action when a property is changed.

The working method to enable adb via Magisk is as follows:

- Boot the phone from a recovery (like TWRP)

- Install the new OS image but do not reboot the phone (see How to install an OS image using the TWRP binary twrp for how to install the OS via the cli commands from TWRP)

- Install Magisk into the boot partition of the phone and delete all files in /data/adb afterwards (see How to install Magisk into the boot partition using a script for how to install Magisk via script)

- copy a script to the directory /data/recovery on the phone that will enable access via adb using the Android command settings and copy the public ssl keys for the access via adb to the directory /data/misc/adb on the phone

- install a new init .rc file via Magisk which executes the script in the direcory /data/recovery to enable the access via adb - this script will be executed while doing the 1st reboot of the new installed Android OS.
(see How to trigger an action when a property is changed for how to add a new init .rc file using Magisk)

- reboot the phone

I wrote two scripts to enable adb using this method:

The script enable_adb_using_magisk.sh must run in an adb session in TWRP after the new OS was installed and after Magisk has been installed into the boot partition but before the first reboot of the new installed operating system.

The script enable_adb_using_magisk.sh

- creates the script /data/recover/enable_adb_via_service.sh on the phone to enable adb and install the public ssl key for access via adb

- creates a new init *rc file using Magisk that will run the script /data/recover/enable_adb_via_service.sh on the phone while doing the 1st reboot (after the file encryption for the files /data is initalized by the Android OS)

To use the script enable_adb_using_magisk.sh manually after installing the new OS and Magisk do while the phone is booted into TWRP:

# # copy the script enable_adb_using_magisk.sh to the directory /tmp on the phone # adb push enable_adb_using_magisk.sh /tmp/enable_adb_using_magisk.sh # copy the public ssl key for access via adb to the directory /tmp on the phone # adb push $HOME/.android/adbkey.pub /tmp/adbkey.pub # execute the script /tmp/enable_adb_using_magisk.sh on the phone # adb shell sh /tmp/enable_adb_using_magisk.sh # reboot the phone into the new installed OS # adb reboot

Notes:

The script enable_adb_using_magisk.sh installs the init *rc file into the boot partition from the slot for the next reboot: In this scenario this is the slot with the new installed OS.

To specify the boot partition to use add the boot partition as parameter, e.g.

adb shell sh /tmp/enable_adb_using_magisk.sh _a

The init *rc service used is:

init *rc service to enable adb

# # additional RC service to enable the access via adb after the 1st boot into the new installed Android OS # service bnsmb_enable_adb /system/bin/sh /data/recovery/enable_adb_via_service.sh user root group root seclabel u:r:magisk:s0 disabled oneshot on zygote-start setprop sys.bnsmb_enable_adb_done 0 start bnsmb_enable_adb # # Note: the following entries are for testing only! # on zygote-start write /data/recovery/semfile Here_I_am setprop sys.bnsmb.test.okay 0

The script enable_adb_using_magisk.sh creates the files adb_keys and adb_temp_keys.xml in the directory /data/misc/adb if the public ssl key /tmp/adbkey.pub exists on the phone.

Note that the file /data/misc/adb/db_temp_keys.xml is an Android Binary XML fIle (see How to change files in Android XML Binary format for details about Android Binary XML files)

To avoid the manual steps listed above I wrote another script:

enable_access_via_adb.sh

The script enable_access_via_adb.sh does all the steps listed above; the usage for enable_access_via_adb.sh is

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./enable_access_via_adb.sh -h enable_access_via_adb.sh version - v1.0.0.0 - shell script to enable adb access via an init .rc file configured via Magisk enable_access_via_adb.sh [-h|help|-H] [--reboot|--noreboot] [--nopubkey|--pubkey] [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Detailed usage

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./enable_access_via_adb.sh -H enable_access_via_adb.sh version - v1.0.0.0 - shell script to enable adb access via an init .rc file configured via Magisk Usage enable_access_via_adb.sh [-h|help|-H] [--reboot|--noreboot] [--nopubkey|--pubkey] All parameter are optional. The parameter can be used in any order. Use the parameter "help" or "-H" to print the detailed usage help; use the parameter "-h" to print only the short usage help If the parameter "--reboot" is used the script will reboot the phone after installing Init .rc file; to disable the automatic reboot use the parameter "--noreboot". Default is to ask the user for confirmation to reboot the phone. Use the parameter "--nopubkey" to disable configuring the public key of the current user on the PC for the access via adb; use the parameter "--pubkey" to configure the public key; default is to configure the public key. The default public ssl key used is the key in the file "${HOME}/.android/adbkey.pub". To change some of the values used by the script these environment variables can be set before starting the script: Set the environment variable PUBLIC_KEY_ON_PC to the file with the public ssl key to use for the access via adb if another public key should be used Set the environment variable REBOOT to "yes" before starting the script to automatically reboot the phone after enabling the root access Set the environment variable REBOOT to "no" before starting the script to disable the automatic reboot after enabling the root access See also the source code of the script boot_phone_from_twrp.sh for environment variables supported by this script Prerequisites - the phone must be connected via USB - Magisk must be already installed in the boot partition of the phone - the phone must be either booted into the fastbootd or bootloader with a working fastboot connection or already booted into a recovery image with working adb connnection The scripts boot_phone_from_twrp.sh and enable_adb_using_magisk.sh are required by this script -- see the source code of the script [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

So installing a new OS and enabling access via adb in a script without user intervention can be done using these commands:

# # do a factory reset and install the OS image using the TRWP binary twrp # REBOOT=no ./install_os_via_twrp.sh factory_reset /data/backup/ASUS_ZENFONE8/omnirom_local/omni-13/omni-13-20221211-zenfone8-MICROG.zip # install Magisk into the boot sector of the partition with the new OS and delete all files in /data/adb afterwards # # The parameter delete_adb_dir for the script "install_magisk_via_twrp.sh" is only available in version 2.2.0.0 or newer of the script. # REBOOT=no ./install_magisk_via_twrp.sh next copy_apk delete_adb_dir # enable adb and copy the public keys for adb via new init *rc file using Magisk in the boot partition # REBOOT=no ./enable_access_via_adb.sh # reboot the phone # adb reboot # wait until the new OS is booted and adb is enabled # adb wait-for-device # install the Magisk App # ./install_apk.sh /data/backup/Android/EssentialApps/Magisk-v25.2.apk # install the Magisk directories and binaries in /data/adb # REBOOT=no ./install_magisk_via_twrp.sh adb_only # (optional) enable root access for the adb shell # REBOOT=yes ./init_magisk_db.sh

Notes:

The script enable_access_via_adb.sh needs the helper script boot_phone_from_twrp.sh and the script enable_adb_using_magisk.sh .

The scripts can be downloaded from my homepage:

enable_access_via_adb.sh

enable_adb_using_magisk.sh

boot_phone_from_twrp.sh

The other scripts mentioned in ths post can also be downloaded from my home page:

install_os_via_twrp.sh (see How to install an OS image using the TWRP binary twrp for a descripton of this script)

install_magisk_via_twrp.sh (see How to install Magisk into the boot partition using a script for a description for this script)

install_apk.sh (see How to install packages (apk files) for Android via script for a description for this script)

init_magisk_db.sh (see How to enable root access using Magisk in a script for a description for this script)

Disabling the service to enable access via adb

To only run once, the script enable_adb_using_magisk.sh creates the file /data/recovery/adb_initialized exists on the phone. If that file already exists when enable_adb_using_magisk.sh starts, the script will do nothing and end immediately.
Therefor, it's not necessary to delete the new init *rc service.

It might even be useful to leave the service installed, since it can be used for other purposes later by modifying the script

However, if necessary, you can either delete the script used by the init *rc service enable_adb_using_magisk.sh or reinstall the original boot partition to get rid of the new service completely:

# # while the phone is booted into the Android OS with enabled root access or while the phone is booted into TWRP # # use the parameter "yes" for the script to run without requesting user input # adb shell su - -c /data/recovery/work/restore_boot_partition.sh

Directories and files used

Name	Type	Content	Comment
/tmp/adbkey.pub	file	ssl public key that should be configured	/tmp is on a ramdisk so this file is lost after booting the phone

/data/recovery/work	Directory	temporary files used for enabling adb
/data/recovery/work/enable_adb_via_service.rc	file	init *rc file added to the boot partition via Magiks
/data/recovery/work/boot_a.img	file	image of the original boot partition
/data/recovery/work/original_boot_partition.img	SymLink	link to the image with the original boot partition
/data/recovery/work/restore_boot_partition.sh	file	Script to restore the boot partition using the image of the original boot partition
/data/recovery/work/header /data/recovery/work/kernel /data/recovery/work/ramdisk.cpio	files	contents of the boot partition

/data/recovery/adbkey.pub	file	public ssl key to be configured for access via adb
/data/recovery/adb_temp_keys.xml.human	file	XML file with the configuration for the access via adb (will be converted to an XML file in Android binary XML format in /data/misc/adb)
/data/recovery/enable_adb_via_service.sh	file	script to enable the access via adb and installl the public ssl key for the access via adb The script is configured in the init*rc file /data/recovery/work/enable_adb_via_service.rc
/data/recovery/enable_adb_via_service.log	file	log file created by the script /data/recovery/enable_adb_via_service.sh
/data/recovery/adb_initialized	file	semphor file for the script /data/recovery/enable_adb_via_service.sh - if this file exists the script will do nothing
/data/recovery/semfile	file	test file created by the new init *rc service

/data/recovery/log.gz /data/recovery/last_log.gz /data/recovery/recovery.fstab /data/recovery/storage.fstab	file	Files created by TWRP	only for information

Enabling access via adb in the LineageOS (Update 11.01.2023)

In the LineageOS 19 for the ASUS Zenfone 8 access via adb is enabled in the default configuration. Therefor these instructions are not necessary. The only configuration that must be done on a phone running the LineageOS is to copy the public ssl keys to /data/misc/adb to enable the access via adb. Unfortunately the method used for this described in this post seems not to work in the LineageOS.
In addition, root access via adb can be enabled in the developer settings in the LineageOS. Until now I did not find out how to enable the plain root access from the LineageOS from within a script.

Trouble Shooting

The script enable_adb_using_magisk.sh uses the log file /data/recovery/enable_adb_via_service.log if executed via init *rc service.

To check if the init *rc file was configured successfully check the properties

sys.bnsmb_enable_adb_done
sys.bnsmb.test.okay

in the running Android OS. Both properties should be defined with the value 0:

ASUS_I006D:/ # getprop sys.bnsmb_enable_adb_done 0 ASUS_I006D:/ # ASUS_I006D:/ # getprop sys.bnsmb.test.okay 0 ASUS_I006D:/ #

In addition, if everything worked there should exist the file /data/recovery/semfile.

ASUS_I006D:/ # ls -l /data/recovery/semfile -rw------- 1 root root 9 2022-12-29 11:44 /data/recovery/semfile ASUS_I006D:/ #

Use the Android command start to check if the service exists and can be started, e.g.

start bnsmb_enable_adb

ASUS_I006D:/ # start bnsmb_enable_adb ; dmesg | tail -10 [ 3069.861365] [ 3069.861365] (CPU:2-pid:1:init) [12:35:31.711593934] init: Service 'exec 613 (/system/bin/flags_health_check UPDATABLE_CRASHING)' (pid 5734) exited with status 0 waiting took 0.024000 seconds [ 3069.861402] [ 3069.861402] (CPU:2-pid:1:init) [12:35:31.711630236] init: Sending signal 9 to service 'exec 613 (/system/bin/flags_health_check UPDATABLE_CRASHING)' (pid 5734) process group... [ 3069.861618] [ 3069.861618] (CPU:2-pid:1:init) [12:35:31.711846069] libprocessgroup: Successfully killed process cgroup uid 1000 pid 5734 in 0ms [ 3071.202534] [ 3071.202486] (CPU:2-pid:5077:wk:0xffffffe40) [12:35:33.052716433] [BAT][CHG]asus_jeita_cc_worker set BATTMAN_OEM_WORK_EVENT : WORK_JEITA_CC [ 3071.220807] [ 3071.220807] (CPU:0-pid:350:wk:0xffffffe40) [12:35:33.071038100] [BAT][CHG]handle_message set property:16 successfully [ 3071.570753] [ 3071.570753] (CPU:2-pid:1:init) [12:35:33.420982631] init: starting service 'bnsmb_enable_adb'... [ 3071.575971] [ 3071.575971] (CPU:3-pid:1:init) [12:35:33.426199819] init: Control message: Processed ctl.start for 'bnsmb_enable_adb' from pid: 5735 (start bnsmb_enable_adb) [ 3071.614570] [ 3071.614570] (CPU:3-pid:1:init) [12:35:33.464799298] init: Service 'bnsmb_enable_adb' (pid 5736) exited with status 0 oneshot service took 0.040000 seconds in background [ 3071.614609] [ 3071.614609] (CPU:3-pid:1:init) [12:35:33.464837267] init: Sending signal 9 to service 'bnsmb_enable_adb' (pid 5736) process group... [ 3071.614788] [ 3071.614788] (CPU:3-pid:1:init) [12:35:33.465016277] libprocessgroup: Successfully killed process cgroup uid 0 pid 5736 in 0ms ASUS_I006D:/ #

If something does not work execute the script manually in an adb session to check :

/data/recovery/enable_adb_via_service.sh

ASUS_I006D:/ # rm -f /data/recovery/adb_initialized ASUS_I006D:/ # ASUS_I006D:/ # ASUS_I006D:/ # sh -x /data/recovery/enable_adb_via_service.sh + tty -s + '[' 0 -ne 0 ']' + PATH=/system/bin:/product/bin:/apex/com.android.runtime/bin:/apex/com.android.art/bin:/system_ext/bin:/system/bin:/system/xbin:/odm/bin:/vendor/bin:/vendor/xbin + export PATH + '[' ! -r /data/recovery/adb_initialized ']' + echo 'Sleeping 30 seconds now ...' Sleeping 30 seconds now ... + sleep 30 + touch /data/recovery/adb_initialized + settings put global development_settings_enabled 1 + settings put global adb_enabled 1 + '[' ! -d /data/misc/adb ']' + '[' ! -r /data/misc/adb/adb_keys ']' + '[' -r /data/recovery/adbkey.pub ']' + >>/data/misc/adb/adb_keys + cat /data/recovery/adbkey.pub + '[' -r /data/recovery/adb_temp_keys.xml.human ']' + xml2abx /data/recovery/adb_temp_keys.xml.human /data/misc/adb/adb_temp_keys.xml + chmod 0600 /data/misc/adb/adb_temp_keys.xml + chown system:shell /data/misc/adb/adb_temp_keys.xml + chcon -v u:object_r:adb_keys_file:s0 /data/misc/adb/adb_temp_keys.xml chcon '/data/misc/adb/adb_temp_keys.xml' to u:object_r:adb_keys_file:s0 ASUS_I006D:/ #

see also How to enable adb via WiFi

Update 03.05.2024

see Some hints about adb and fastboot usage in Linux for more detailed infos about the SSL keys used for the adb connection

Some hints about the Android safe mode

URL: https://xdaforums.com/t/some-hints-about-the-android-safe-mode.4536269/

Some hints about the Android safe mode

Safe mode is a feature in Android phones where all third-party apps on the device become inaccessible when enabled.

Doing a Google search on Android safe mode, I found a lot of information that is outdated and no longer applies to the current Android version.

So this small HowTo contains some infos about using the Android safe mode in Android 13 ; the infos should also be valid for Android 12 but I did not test that yet.

How to check if the phone is booted into safe mode in the Android GU

In safe mode Android prints the string "Safe mode" in the lower left corner

How to check if the phone is booted into safe mode in a shell as root user

The root user on the phone can check the value of the property persist.sys.safemode; this property is set to 1 in safe mode and does not exist if not in safe mode

in safe mode:

ASUS_I006D:/ # id uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 ASUS_I006D:/ # ASUS_I006D:/ # getprop persist.sys.safemode 1 ASUS_I006D:/ #
in normal mode:

ASUS_I006D:/ # id uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 ASUS_I006D:/ # ASUS_I006D:/ # getprop persist.sys.safemode ASUS_I006D:/ #

How to check if the phone is booted into safe mode in a shell as non-root user

Non-root user can not read the property persist.sys.safemode. Non-root user can use the output of the command dumpsys to check if the phone is booted into safe mode using this command:

dumpsys display | grep mSafeMode

e.g.:

in safe mode:

ASUS_I006D:/ $ id uid=2000(shell) gid=2000(shell) groups=2000(shell),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),1078(ext_data_rw),1079(ext_obb_rw),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats),3009(readproc),3011(uhid),3012(readtracefs) context=u:r:shell:s0 ASUS_I006D:/ $ ASUS_I006D:/ $ dumpsys display | grep mSafeMode mSafeMode=true ASUS_I006D:/ $
in normal mode;

ASUS_I006D:/ $ id uid=2000(shell) gid=2000(shell) groups=2000(shell),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),1078(ext_data_rw),1079(ext_obb_rw),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats),3009(readproc),3011(uhid),3012(readtracefs) context=u:r:shell:s0 ASUS_I006D:/ $ ASUS_I006D:/ $ dumpsys display | grep mSafeMode mSafeMode=false ASUS_I006D:/ $

How to enable the safe mode for the next reboot in the Android GUI

To enable safe mode via the Android GUI do :

Presse the power button until the reboot dialog with Power Off, Restart, etc appears. Now long tap on the PowerOff button in that dialog until the dialog to reboot into safe mode appears and press ok in that dialog. The phone will then be booted into safe mode.

How to enable the safe mode for the next reboot via shell command

To enable safe mode for the next reboot of the Android OS when in the running Android OS open a shell (either local or via adb), become root user and, issue

setprop persist.sys.safemode 1 reboot

Note that Android will automatically clear this property after booting into the normal mode again

How to enable the safe mode for the next reboot when booted into the recovery

The current value for the property persist.sys.safemode is stored in the file

/data/property/persistent_properties

e.g:

ASUS_I006D:/ # grep safe /data/property/persistent_properties persist.sys.safemode ASUS_I006D:/ #

Unfortunately this is a binary file with an unknown format:

ASUS_I006D:/ # file /data/property/persistent_properties /data/property/persistent_properties: data ASUS_I006D:/ #
So it's not clear how to enable safe mode for the next reboot while booted in to recovery.

A very ugly workaround to enable safe mode for the next reboot while the phone is booted from the recovery can be implemented using a Magisk init script (if Magisk is installed on the phohe, of course - see How to run a script at every boot using Magisk for details):

ASUS_I006D:/ # cat /data/adb/post-fs-data.d/enable_safe_mode /data/adb/magisk/magisk64 resetprop -n persist.sys.safemode 1 # # make sure booting into safe mode is only a onetime job! # rm /data/adb/post-fs-data.d/enable_safe_mode ASUS_I006D:/ #

Notes:

Using this method the propery persist.sys.safemode is not set after booting into safe mode and the safe mode can only be checked using the output of the dumpsys command, e.g.:

ASUS_I006D:/ # id uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 ASUS_I006D:/ # ASUS_I006D:/ # getprop persist.sys.safemode ASUS_I006D:/ # ASUS_I006D:/ # dumpsys display|grep mSafeMode mSafeMode=true ASUS_I006D:/ #

Be aware that without deleting the Magisk init script for the workaround the phone will always boot into safe mode.

How to exit the safe mode

To exit the safe mode Just reboot the phone

How to disable safe mode

Disabling safe mode is only possible using an Android MDM (= Mobile Device Manager). For further details see this thread :

https://xdaforums.com/t/blocking-safe-mode-and-factory-reset-on-android.4603707/

and/or these apps:

Sentry

https://github.com/x13a/Sentry?tab=readme-ov-file

The Sentry app is available via F-Droid; I successfully tested disabling safe mode using this app on an ASUS Zenfone 8 running OmniROM 14 (= Android 14)

Test DPC from Google to test Android Device Management (not tested)

https://github.com/googlesamples/android-testdpc

Misc

The network is not configured in safe mode

Magisk is disabled in safe mode

History of this entry

19.01.2024

added some misc. infos about safe mode

08.02.2024

added infos about how to disable safe mode

How to install and configure the Android OS via Script

URL: https://xdaforums.com/t/how-to-install-and-configure-the-android-os-via-script.4546375/

How to install and configure the Android OS via Script

Note:

The instructions below work in my environment - they should work for others after adjusting the configuration. But of course I can't guarantee that.

Update 22.05.2024 /bs

I have updated the information in this section to reflect any changes and improvements I have made in the time since writing this article

Update 01.05.2024/bs

The instructions in this entry only work for ROM images that can be installed using the twrp binary from TWRP with the parameter install (for example the OmniROM).

See the entry How to install and configure other ROMs via script for instructions to install the original Android from ASUS or a LineageOS based OS using the scripts described in this entry.

After creating all the other scripts to install and configure the Android OS without user intervention I finally wrote a script to do all the steps necessary to install and configure the Android OS using these scripts.

For this purpose I used another script that I wrote some time ago and use also for a lot of other things: execute_tasks.sh.

execute_tasks.sh reads a list of defined tasks from one or more include files and executes one or more of these tasks one after the other. One or more parameter for the tasks are supported. It's also possible to define task groups with one or more tasks that can be used like single tasks.

The usage for execute_tasks.sh is :

./execute_tasks.sh -h

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ date Thu May 23 12:25:23 PM CEST 2024 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./execute_tasks.sh -h [23.05.2024 12:25 ] ### execute_tasks.sh started at 23.05.2024 12:25:24 (The PID of this process is 555399) [23.05.2024 12:25 ] ### Processing the parameter ... [23.05.2024 12:25 ] Tasks to execute are: "" [23.05.2024 12:25 ] Parameter for the function init_tasks are: "" Usage: execute_tasks.sh [-v|--verbose] [-q|--quiet] [-f|--force] [-o|--overwrite] [-y|--yes] [-n|--no] [-l|--logfile filename] [-d{:dryrun_prefix}|--dryrun{:dryrun_prefix}] [-D|--debugshell] [-t fn|--tracefunc fn] [-L] [-T|--tee] [-V|--version] [--var name=value] [--appendlog] [--nologrotate] [--noSTDOUTlog] [--disable_tty_check] [--nobackups] [--print_task_template [filename]] [--create_include_file_template [filename]] [--list [taskmask]] [--list_tasks [taskmask]] [--list_task_groups [groupmask]] [--list_default_tasks] [--abort_on_error] [--abort_on_task_not_found] [--abort_on_duplicates] [--checkonly] [--check] [--singlestep] [--singlestep_on_error] [--unique] [--trace] [--info] [--print_includefile_help] [-i|--includefile [?|+]filename] [--no_init_tasks[ [--no_finish_tasks] [--only_list_tasks] [--disabled_tasks task1[...,task#]] [--list_disabled_tasks] [--enable_all_tasks] [task1] [... task#] [-- parameter_for_init_tasks] Current environment: ksh version: 93 | change function code supported: yes | tracing feature using $0 supported: yes [23.05.2024 12:25 ] ### The start time was 23.05.2024 12:25:24, the script runtime is (day:hour:minute:seconds) 0:00:00:00 (= 0 seconds) [23.05.2024 12:25 ] ### execute_tasks.sh ended at 23.05.2024 12:25:24 (The PID of this process is 555399; the RC is 0) [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

To get a more detailed usage description either download the script and execute it with the parameter -H or use this link .

The include file with the tasks to install and configure the Android OS, prepare_phone.include, can be downloaded here.

To use that file either use

./execute_tasks.sh -i prepare_phone.include [task1 ... task#]

or create the symbolic link prepare_phone.sh for execute_tasks.sh :

ln -s ./execute_tasks.sh ./prepare_phone.sh

and use it like this

./prepare_phone.sh [task1 ... task#]

Notes:

prepare_phone.sh and prepare_phone.include must be in the same directory. In the rest of this post I use this method.

prepare_phone.sh first checks whether a phone is connected via USB. If you are performing a task that does not require a connected phone, add the parameter " -- nophonecheck" to avoid an error if no phone is connected to the PC

To list all defined tasks in the include file prepare_phone.include use the parameter --list, e.g:

./prepare_phone.sh --list -v

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ date Wed May 22 01:53:51 PM CEST 2024 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./prepare_phone.sh --list -v [22.05.2024 13:53 ] ### prepare_phone.sh started at 22.05.2024 13:53:58 (The PID of this process is 157710) [22.05.2024 13:53 ] ### Processing the parameter ... [22.05.2024 13:53 ] Tasks to execute are: "" [22.05.2024 13:53 ] Parameter for the function init_tasks are: "" [22.05.2024 13:53 ] ### The logfile used is /var/tmp/prepare_phone.sh.log [22.05.2024 13:53 ] INFO: Searching the default include file "prepare_phone.include" ... [22.05.2024 13:53 ] INFO: ... "/data/develop/android/scripts_on_linux/prepare_phone.include" found. [22.05.2024 13:53 ] INFO: Checking and reading the include file(s) "/data/develop/android/scripts_on_linux/prepare_phone.include " now ... [22.05.2024 13:53 ] Checking the include file "/data/develop/android/scripts_on_linux/prepare_phone.include" ... [22.05.2024 13:53 ] INFO: The version of the include file "/data/develop/android/scripts_on_linux/prepare_phone.include", "1.0.0.0", is okay. [22.05.2024 13:53 ] INFO: Function "init_tasks" found in the include file "/data/develop/android/scripts_on_linux/prepare_phone.include" [22.05.2024 13:53 ] INFO: Function "finish_tasks" found in the include file "/data/develop/android/scripts_on_linux/prepare_phone.include" [22.05.2024 13:53 ] Reading the include file "/data/develop/android/scripts_on_linux/prepare_phone.include" ... [22.05.2024 13:53 ] INFO: Found this list of tasks to be excluded from "all" in the include file "/data/develop/android/scripts_on_linux/prepare_phone.include" : [22.05.2024 13:53 ] INFO: Found this list of tasks for "all" in the include file "/data/develop/android/scripts_on_linux/prepare_phone.include" : [22.05.2024 13:53 ] INFO: enable_disable_abort_on_error:enable [22.05.2024 13:53 ] INFO: select_twrp_image:ignorerc [22.05.2024 13:53 ] INFO: boot_phone_from_twrp_image:force [22.05.2024 13:53 ] INFO: install_os:noreboot:factory_reset [22.05.2024 13:53 ] INFO: restart_adb_daemon [22.05.2024 13:53 ] INFO: backup_boot_partition:next [22.05.2024 13:53 ] INFO: install_twrp:noreboot:next [22.05.2024 13:53 ] INFO: install_magisk_in_boot_partition_only:noreboot:next [22.05.2024 13:53 ] INFO: enable_adb:reboot [22.05.2024 13:53 ] INFO: execute_script_on_the_phone:ignorerc:pre_install.sh [22.05.2024 13:53 ] INFO: wait_for_access_via_adb:240 [22.05.2024 13:53 ] INFO: task_select_twrp_image:ignorerc [22.05.2024 13:53 ] INFO: install_magisk_in_data_adb_only:noreboot:copy_apk:use_apk [22.05.2024 13:53 ] INFO: wait_for_access_via_adb:120 [22.05.2024 13:53 ] INFO: enable_root_access_for_the_shell:reboot [22.05.2024 13:53 ] INFO: wait_for_access_via_adb:120 [22.05.2024 13:53 ] INFO: install_magisk_v26_in_boot_partition:active:reboot [22.05.2024 13:53 ] INFO: wait_for_access_via_adb:120 [22.05.2024 13:53 ] INFO: install_the_magisk_app [22.05.2024 13:53 ] INFO: install_archive_file:twrp:reboot:script=/cache/correct_setlinux_for_wifconifg.sh [22.05.2024 13:53 ] INFO: enable_disable_abort_on_error:disable [22.05.2024 13:53 ] INFO: install_essential_scripts [22.05.2024 13:53 ] INFO: execute_script_on_the_phone:ignorerc:/data/develop/android/scripts_on_linux/enable_wlan.sh [22.05.2024 13:53 ] INFO: install_essential_apps:ignorerc [22.05.2024 13:53 ] INFO: install_essential_magisk_modules [22.05.2024 13:53 ] INFO: install_magisk_modules:${MICROG_MAGISK_MODULE}:ignorerc [22.05.2024 13:53 ] INFO: enable_root_access_for_apps:reboot [22.05.2024 13:53 ] INFO: execute_script_on_the_phone:ignorerc [22.05.2024 13:53 ] INFO: wait_for_access_via_adb Include files used are: /data/develop/android/scripts_on_linux/prepare_phone.include Tasks defined are: Task: backup_boot_partition Usage: backup_boot_partition [slot] [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [twrp] [notwrp] [twrp=twrp_image] [image_file_on_the_phone] [image_file_on_the_pc] Task: boot_phone_from_twrp_image Usage: boot_phone_from_twrp_image [ignorerc] [twrp_image] [force] [reboot] Task: boot_phone_into_android Usage: boot_phone_into_android [ignorerc] [force] Task: boot_phone_into_bootloader Usage: boot_phone_into_bootloader [ignorerc] [force] Task: boot_phone_into_fastboot Usage: boot_phone_into_fastboot [ignorerc] [force] Task: boot_phone_into_recovery Usage: boot_phone_into_recovery [ignorerc] [force] Task: boot_phone_into_safemode Usage: boot_phone_into_safemode [ignorerc] [force] [timeout=timeout_in_seconds] # Default: 30 seconds Task: check_config Usage: check_config Task: connect_using_adb_via_wifi Usage: connect_using_adb_via_wifi [testonly] [ip=IP] [port=PORT] [localport=PORT] [ignorerc] Task: copy_file_to_phone Usage: copy_file_to_phone [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [source_file] [target_file|target_dir] Task: DebugShell Usage: DebugShell Task: decrypt_data_partition Usage: decrypt_data_partition [ignorerc] [password] Task: disable_adb_via_wifi Usage: disable_adb_via_wifi [restart_adbd|restart_adbd=yes|restart_adbd=no] [check_port|check_port=yes|check_port=no] [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] Task: download_scripts Usage: download_scripts [target_dir|default|pwd] [backup|nobackup] [ignorerc] # default: download the scripts to the current directory Task: enable_adb Usage: enable_adb [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [slot] [parameter_for_the_script] # default is: no reboot, inactive slot Task: enable_adb_via_wifi Usage: enable_adb_via_wifi [port] [restart_adbd|restart_adbd=yes|restart_adbd=no] [check_port|check_port=yes|check_port=no] [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] Task: enable_disable_abort_on_error Usage: enable_disable_abort_on_error [yes|no] # default is yes Task: enable_root_access_for_apps Usage: enable_root_access_for_apps [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [parameter_for_the_script] # default is: no reboot Task: enable_root_access_for_the_shell Usage: enable_root_access_for_the_shell [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [parameter_for_the_script] # default is: no reboot Task: enable_single_step_mode Usage: enable_single_step_mode [on_error] Task: execute_command_on_the_phone Usage: execute_command_on_the_phone [ignorerc] [command_to_execute] # no default, no whitepspaces allowed; execute w/o task parameter to open an interactive shell Task: execute_script_on_the_phone Usage: execute_script_on_the_phone [ignorerc] [script_to_execute] # default: post_install.sh in the current directory Task: factory_reset_via_twrp Usage: factory_reset_via_twrp [reboot|noreboot|reboot=no|reboot=yes] [twrp_imagefile] [ignorerc] [force|noforce] Task: format_data_via_twrp Usage: format_data_via_twrp [reboot|noreboot|reboot=no|reboot=yes] [twrp_imagefile] [ignorerc] [force|noforce] Task: format_metadata_via_twrp Usage: format_metadata_via_twrp [reboot|noreboot|reboot=no|reboot=yes] [twrp_imagefile] [ignorerc] [force|noforce] Task: install_apps Usage: install_apps [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [apkfile1 ... apkfile#] # default is: no reboot Task: install_archive_file Usage: install_archive_file [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [twrp] [reocvery] [android] [force] [file] [script=scriptname] Task: install_ASUS_raw_image Usage: install_ASUS_raw_image [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [dir_with_raw_image] [paremeter_for_the_install_script] Task: install_essential_apps Usage: install_essential_apps [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [dir_with_the_apk_files_to_install] # default is: no reboot Task: install_essential_magisk_modules Usage: install_essential_magisk_modules [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [dir_with_the_magisk_modules_to_install] # default is: no reboot Task: install_essential_scripts Usage: install_essential_scripts [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [dir_with_the_scripts_to_install] # default is: no reboot Task: install_magisk_in_boot_partition Usage: install_magisk_in_boot_partition [slot] [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [magisk_apkfile] [parameter_for_the_script] # default is: reboot Task: install_magisk_in_boot_partition_only Usage: install_magisk_in_boot_partition_only [slot] [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [magisk_apkfile] [parameter_for_the_script] # default is: no reboot Task: install_magisk_in_data_adb_only Usage: install_magisk_in_data_adb_only [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [magisk_apkfile] [parameter_for_the_script] # default is: no reboot Task: install_magisk_modules Usage: install_magisk_modules [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [magisk_module1 ... magisk_module#] # default is: no reboot Task: install_magisk_v26_in_boot_partition Usage: install_magisk_v26_in_boot_partition [slot] [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [magisk_apkfile] [parameter_for_the_script] # default is: reboot Task: install_os Usage: install_os [slot] [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [reset] [imagefile] [parameter_for_the_script] # default is: no reboot Task: install_the_magisk_app Usage: install_the_magisk_app [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [magisk_apk_file] # default is: no reboot Task: install_twrp Usage: install_twrp [slot] [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [twrp_imagefile] [parameter_for_the_script] # default is: no reboot Task: kill_adb_daemon Usage: kill_adb_daemon [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] Task: list_taskgroups Usage: list_taskgroups Task: mount_data_via_twrp Usage: mount_data_via_twrp [reboot|noreboot|reboot=no|reboot=yes] [twrp_imagefile] [ignorerc] [force|noforce] Task: reset_usb_port Usage: reset_usb_port [reboot|reboot=no|reboot=yes] [ignorerc] [force] [usb_port] [...] Task: restart_adb_daemon Usage: restart_adb_daemon [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [root] Task: restart_adbd_on_the_phone Usage: task_restart_adbd_on_the_phone [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] Task: restore_boot_partition Usage: restore_boot_partition [slot] [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [twrp] [twrp=twrp_image] [[pc=|phone=]image_file] Task: restore_titanium_backup Usage: restore_titanium_backup [license=license_file] [zip=zipfile] [app=titanium_app] [ignorerc] Task: select_twrp_image Usage: select_twrp_image Task: select_twrp_image_for_install_image Usage: select_twrp_image_for_install_image [ignorerc] [image_file] # default Task: set_os_image_to_install Usage: set_os_image_to_install [os_image] # there is no default value Task: set_twrp_image_to_use Usage: set_twrp_image_to_use [force] [twrp_image|default|lineageos] # there is no default value Task: start_adb_daemon Usage: start_adb_daemon [reboot|noreboot|reboot=no|reboot=yes] [ignorerc] [root] Task: umount_data_via_twrp Usage: umount_data_via_twrp [reboot|noreboot|reboot=no|reboot=yes] [twrp_imagefile] [ignorerc] [force|noforce] Task: update_include_file Usage: update_include_file [target_dir|target_file] [ignorerc] [keep|nokeep] # default: nokeep (= delete temporary files) Task: wait_for_access_via_adb Usage: wait_for_access_via_adb [ignorerc] [kill] [[timeout=]timeout_in_seconds] # default: 30 seconds Task: wait_for_phone_state Usage: wait_for_phone_state [ignorerc] [state|state=#] [timeout=timeout_in_seconds] # default: 3 30 ; use 0 for infinite; set TIMEOUT_VALUE to define the timeout value via env variable Task: wait_n_seconds Usage: wait_n_seconds [wait_time] [step_count] # default for wait_time is 60 and default for step count is 5 Task: wipe_cache_via_twrp Usage: wipe_cache_via_twrp [reboot|noreboot|reboot=no|reboot=yes] [twrp_imagefile] [ignorerc] [force|noforce] Task: wipe_dalvik_and_cache_via_twrp Usage: wipe_dalvik_and_cache_via_twrp [reboot|noreboot|reboot=no|reboot=yes] [twrp_imagefile] [ignorerc] [force|noforce] Task: wipe_dalvik_via_twrp Usage: wipe_dalvik_via_twrp [reboot|noreboot|reboot=no|reboot=yes] [twrp_imagefile] [ignorerc] [force|noforce] Task: wipe_data_via_twrp Usage: wipe_data_via_twrp [reboot|noreboot|reboot=no|reboot=yes] [twrp_imagefile] [ignorerc] [force|noforce] 61 task(s) defined. Defined Task groups are: abort_on_error : enable_disable_abort_on_error:enable all_no_magisk_no_twrp : enable_disable_abort_on_error:enable select_twrp_image:ignorerc boot_phone_from_twrp_image install_os:noreboot:factory_reset:auto kill_adb_daemon backup_boot_partition:next install_magisk_in_boot_partition_only:noreboot:next enable_adb:reboot execute_script_on_the_phone:ignorerc:pre_install.sh wait_for_access_via_adb:240:kill select_twrp_image:ignorerc install_magisk_in_data_adb_only:noreboot:copy_apk:use_apk wait_for_access_via_adb:120:kill enable_root_access_for_the_shell:reboot wait_for_access_via_adb:120:kill install_archive_file:twrp:reboot:script=/cache/correct_setlinux_for_wifconifg.sh enable_disable_abort_on_error:disable execute_script_on_the_phone:ignorerc:/data/develop/android/scripts_on_linux/enable_wlan.sh restore_boot_partition:twrp:noreboot install_essential_apps execute_script_on_the_phone:ignorerc all_sideload : enable_disable_abort_on_error:enable select_twrp_image:ignorerc boot_phone_from_twrp_image install_os:noreboot:sideload:factory_reset:restart_as_root select_twrp_image_for_install_image boot_phone_from_twrp_image:force backup_boot_partition:active install_magisk_in_boot_partition_only:noreboot:active copy_file_to_phone:/data/backup/Android/EssentialApps/com.termoneplus_501.apk:/data/recovery:ignorerc copy_file_to_phone:/data/backup/Android/EssentialApps/com.jecelyin.editor.apk:/data/recovery:ignorerc copy_file_to_phone:/data/backup/Android/EssentialApps/com.offsec.nhterm_2020040001.apk:/data/recovery:ignorerc enable_adb:reboot:active execute_script_on_the_phone:ignorerc:pre_install.sh install_magisk_in_data_adb_only:noreboot:copy_apk:use_apk wait_for_access_via_adb:120 enable_root_access_for_the_shell:reboot wait_for_access_via_adb:120 install_magisk_v26_in_boot_partition:active:reboot wait_for_access_via_adb:120 install_the_magisk_app wait_for_access_via_adb:120 install_archive_file:twrp:reboot:script=/cache/correct_setlinux_for_wifconifg.sh enable_disable_abort_on_error:disable install_essential_scripts execute_script_on_the_phone:ignorerc:/data/develop/android/scripts_on_linux/enable_wlan.sh install_essential_apps:/data/backup/Android/Test_EssentialApps install_essential_magisk_modules:/data/backup/Android/Test_EssentialMagiskModules enable_root_access_for_apps:reboot execute_script_on_the_phone:ignorerc all_test : enable_disable_abort_on_error:enable select_twrp_image:ignorerc boot_phone_from_twrp_image:force install_os:noreboot:factory_reset restart_adb_daemon backup_boot_partition:next install_twrp:noreboot:next install_magisk_in_boot_partition_only:noreboot:next enable_adb:reboot execute_script_on_the_phone:ignorerc:pre_install.sh wait_for_access_via_adb:240:kill task_select_twrp_image:ignorerc install_magisk_in_data_adb_only:noreboot:copy_apk:use_apk wait_for_access_via_adb:120:kill enable_root_access_for_the_shell:reboot wait_for_access_via_adb:120:kill install_magisk_v26_in_boot_partition:active:reboot wait_for_access_via_adb:120:kill install_the_magisk_app install_archive_file:twrp:reboot:script=/cache/correct_setlinux_for_wifconifg.sh enable_disable_abort_on_error:disable install_essential_scripts execute_script_on_the_phone:ignorerc:/data/develop/android/scripts_on_linux/enable_wlan.sh install_essential_apps:/data/backup/Android/Test_EssentialApps install_essential_magisk_modules:/data/backup/Android/Test_EssentialMagiskModules install_magisk_modules:${MICROG_MAGISK_MODULE} enable_root_access_for_apps:reboot execute_script_on_the_phone:ignorerc all_without_os : enable_disable_abort_on_error:enable select_twrp_image:ignorerc execute_script_on_the_phone:ignorerc:pre_install.sh boot_phone_from_twrp_image install_twrp:noreboot:next install_magisk_in_boot_partition_only:noreboot:next enable_adb:reboot install_magisk_in_data_adb_only:noreboot:copy_apk enable_root_access_for_the_shell:reboot wait_for_access_via_adb:120:kill install_magisk_v26_in_boot_partition:active:reboot wait_for_access_via_adb:120:kill install_the_magisk_app wait_for_access_via_adb:120:kill decrypt_data_partition enable_disable_abort_on_error:disable install_archive_file:twrp:reboot install_essential_scripts install_essential_apps install_essential_magisk_modules enable_root_access_for_apps:reboot decrypt_data_partition execute_script_on_the_phone:ignorerc install_magisk : select_twrp_image:ignorerc install_the_magisk_app install_magisk_in_boot_partition:current wait_for_access_via_adb:60:kill enable_root_access_for_the_shell:reboot wait_for_access_via_adb:60:kill install_essential_magisk_modules:reboot no_abort_on_error : enable_disable_abort_on_error:disable os_twrp_magisk : enable_disable_abort_on_error:enable select_twrp_image:ignorerc boot_phone_from_twrp_image install_os:noreboot:factory_reset install_twrp:noreboot:next install_magisk_in_boot_partition_only:noreboot:next enable_adb:reboot execute_script_on_the_phone:ignorerc:pre_install.sh install_magisk_in_data_adb_only:noreboot:copy_apk:use_apk wait_for_access_via_adb:120:kill enable_root_access_for_the_shell:reboot wait_for_access_via_adb:120:kill install_the_magisk_app wait_for_access_via_adb:120:kill install_archive_file:twrp:reboot execute_script_on_the_phone:ignorerc install_magisk_modules:ignorerc:/data/backup/Android/EssentialMagiskModules/twrp-keep.zip raw_image_with_magisk : install_ASUS_raw_image boot_phone_from_twrp_image:ignorerc format_data_via_twrp boot_phone_from_twrp_image:force install_magisk_in_boot_partition_only:noreboot:_a enable_adb:reboot:_a install_magisk_in_data_adb_only:noreboot:copy_apk:use_apk wait_for_access_via_adb:120:kill enable_root_access_for_the_shell:reboot wait_for_access_via_adb:120:kill install_the_magisk_app refresh : update_include_file refresh_all : download_scripts:default update_include_file reset_and_install_os : select_twrp_image:ignorerc install_os:reboot:factory_reset:auto update_os : enable_disable_abort_on_error:enable select_twrp_image:ignorerc boot_phone_from_twrp_image install_os:noreboot install_twrp:noreboot:next install_magisk_in_boot_partition_only:reboot:next wait_for_access_via_adb:120:kill install_magisk_v26_in_boot_partition:active:reboot update_os_w_Magisk25 : select_twrp_image:ignorerc enable_disable_abort_on_error:enable boot_phone_from_twrp_image install_os:noreboot install_twrp:noreboot:next install_magisk_in_boot_partition_only:reboot:next wait_for_androidos : wait_for_phone_state:3 wait_for_bootloader : wait_for_phone_state:4 wait_for_fastboot : wait_for_phone_state:5 wait_for_lineageos_recovery : wait_for_phone_state:7 wait_for_safemode : wait_for_phone_state:6 wait_for_sideload : wait_for_phone_state:8 wait_for_twrp_image : wait_for_phone_state:1 wait_for_twrp_recovery : wait_for_phone_state:2 Note: use ":" to separate the task name and the parameter [22.05.2024 13:53 ] ### The logfile used was /var/tmp/prepare_phone.sh.log [22.05.2024 13:53 ] ### The start time was 22.05.2024 13:53:58, the script runtime is (day:hour:minute:seconds) 0:00:00:00 (= 0 seconds) [22.05.2024 13:53 ] ### prepare_phone.sh ended at 22.05.2024 13:53:58 (The PID of this process is 157710; the RC is 0) [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

The tasks in the include files use the other scripts I wrote to install and configure the Android OS. Therefor you should first download the current version of these scripts either manual (see below) or using the task download_scripts:

./prepare_phone.sh download_scripts

The task download_scripts downloads all necessary scripts from https://bnsmb.de/files/public/Android to the current directory; to download the scripts to another directory add the target directory as task parameter, e.g. to download the scripts into the directory /var/tmp/scripts use:

./prepare_phone.sh download_scripts:/var/tmp/scripts

Next either edit the default values for the various variables in the file prepare_phone.include or, better, create the config file prepare_phone.conf with the necessary config entries like this:

cat ./prepare.conf

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ date Wed May 22 01:56:14 PM CEST 2024 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ cat prepare_phone.conf # --------------------------------------------------------------------- # # config file for prepare_phone.include # # This file can be used to define the global variables used by the tasks in the file prepare_phone.include # # The format of the entries in this file is # # <varname>="<var_value>" # # empty lines and lines starting with a hash "#" will be ignored # # use "ksh -x -n prepare_phone.conf" to check this file for syntax errors # # all environment variables defined in this file with a name starting in column 1 will be exported; i # to define a variable that should NOT be exported, use one or more leading whitespaces # # The serial number of the phone (the variable is only necessary if the phone is not attached while starting the script) # #SERIAL_NUMBER="M6AIB760D0939LX" # the function to detect the phone status should reset the USB port if necessary # RESET_THE_USB_PORT=${__FALSE} # boot_phone_from_twrp.sh should always print status messages # # PRINT_STATUS=${__TRUE} # # --------------------------------------------------------------------- # default password for the encryption in Android (do not change for the current Android versions) # # export USER_PASSWORD="default_password" # --------------------------------------------------------------------- # global variables used in prepare_phone.include # SCRIPT_DIR="/data/develop/android/scripts_on_linux" # ------------------ # # default TWRP image for all ROMS without a specific TWRP image # DEFAULT_TWRP_IMAGE="/data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-1-I006D-enhanced.img" # ------------------ # # initial TWRP image used by the script # TWRP_IMAGE="${TWRP_IMAGE:=${DEFAULT_TWRP_IMAGE}}" # ------------------ # TWRP images for specific ROM image files # # each line must contain these fields (the field separator is a colon ":"; lines starting with a hash "#" are ignored) : # # regex for the image file : TWRP image to use : Name of the ROM : description # # The fields for the name of the ROM and the description are optional # # The regex can only contain the joker characters "?" and "*" # TWRP_IMAGES_FOR_IMAGE_FILES=" # UL-ASUS* : ${DEFAULT_TWRP_IMAGE} : ASUS Android LMODroid* : /data/backup/ASUS_ZENFONE8/LMODroid/twrp_LMODroid-4.2-20240429-RELEASE-sake.img : LMODroid : e-1.21* : /data1/backup/ASUS_ZENFONE8/e/twrp_recovery-e-1.21-t-20240325389105-dev-sake.img : /e/ lineage-20* : /data/backup/ASUS_ZENFONE8/Lineage-20/twrp_3.7.0_12-1-I006D_for_lineageOS20-20240423-sake.img : LineageOS 20.x lineage-21* : /data/backup/ASUS_ZENFONE8/Lineage-21/twrp_3.7.0_12-1-I006D_for_lineageOS21-20240220-sake.img : LineageOS 21.x statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE.zip : /data/backup/ASUS_ZENFONE8/Statix/20240106/twrp_statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE.img : StatixOS statix_sake-20231224-14-v7.1-UPSIDEDOWNCAKE.zip : /data/backup/ASUS_ZENFONE8/Statix/20231229/statix_sake-20231224_twrp.img : StatixOS omni* : ${DEFAULT_TWRP_IMAGE} : OmniROM # " # ------------------ # TWRP images for specific ROMs currently running on the phone # # each line must contain these fields (the field separator is a colon ":"; lines starting with a hash "#" are ignored) : # # property : value : TWRP image to use : Name of the ROM : description # # Use "*" for the field "value" if the property must be defined but the value is meaningless # # The fields for the name of the ROM and the description are optional # # TWRP_IMAGES_FOR_THE_RUNNING_OS=" # vendor.asus.build.ext.version : * : ${DEFAULT_TWRP_IMAGE} : ASUS Android ro.lineage.build.version : 20* : /data/backup/ASUS_ZENFONE8/Lineage-20/twrp_3.7.0_12-1-I006D_for_lineageOS20-20240423-sake.img : LineageOS ro.lineage.build.version : 21* : /data/backup/ASUS_ZENFONE8/Lineage-21/twrp_3.7.0_12-1-I006D_for_lineageOS21-20240220-sake.img : LineageOS ro.statix.version : * : /data/backup/ASUS_ZENFONE8/Statix/20240106/twrp_statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE.img : StatixOS ro.product.system.name : lmodroid_sake : /data/backup/ASUS_ZENFONE8/LMODroid/twrp_LMODroid-4.2-20240429-RELEASE-sake.img : LMODroid ro.omni.version : * : ${DEFAULT_TWRP_IMAGE} : OmniROM ro.build.description : e_sake* : /data/backup/ASUS_ZENFONE8/e/twrp_recovery-e-1.21-t-20240325389105-dev-sake.img : /e/ " # ------------------ # OS: OmniROM 13 # #OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/omnirom/omni-13/omni-13-202308130136-zenfone8-WEEKLY.zip" # ------------------ # OS: OmniROM 14 # #OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/omnirom_local/omni-14/omni-14-20240424-zenfone8-MICROG.zip" #OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/omnirom_local/omni-14/omni-14-20240504-zenfone8-MICROG.zip" OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/omnirom_local/omni-14/omni-14-20240509-zenfone8-MICROG.zip" # #OS_IMAGE_TO_INSTALL="/data/develop/android/OmniROM_14.0/out/target/product/zenfone8/omni-14-20240513-zenfone8-MICROG.zip" #OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/omnirom_local/omni-14/omni-14-20240513-zenfone8-MICROG_with_preinstalled_apps.zip" #OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/omnirom_local/omni-14/omni-14-20240419-zenfone8-MICROG-eng.zip" # ------------------ # OS: ASUS Android 13 # #OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/ASUS_firmware/Android_13/UL-ASUS_I006D-ASUS-33.0210.0210.296-1.1.26-2305-user.zip" #OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/ASUS_firmware/Android_13/UL-ASUS_I006D-ASUS-33.0210.0210.210-1.1.26-2301-user.zip" # ------------------ # OS: LMODroid # #OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/LMODroid/LMODroid-4.2-20240429-RELEASE-sake.zip" # ------------------ # OS: /e/ # #OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/e/e-1.21-t-20240325389105-dev-sake.zip" # ------------------ # OS: LineageOS 19.x # #OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/Lineage-19_Android12/lineage-19.1-20230414-nightly-sake-signed.zip" # ------------------ # OS: LineageOS 20.x # #OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/Lineage-20/lineage-20.0-20240423-nightly-sake-signed.zip" # ------------------ # OS: StatiXOS 7.x (Android 14) # #OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/Statix/20231229/statix_sake-20231224-14-v7.1-UPSIDEDOWNCAKE.zip" # ------------------ # OS: StatiXOS 7.x (Android 14) # #OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/Statix/20240106/statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE.zip" # ------------------ # OS: LineageOS 21.x # #OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/Lineage-21/lineage-21.0-20240220-UNOFFICIAL-KSU-GPU_UV-sake.zip" # ------------------ # # default raw image to be installed # DEFAULT_CUR_RAW_IMAGE_DIR="/data/backup/ASUS_ZENFONE8/raw_images/Android13/OPEN-ZS590KS-32.0501.0403.4-BSP-2206-user-20220705" # Magisk APK files # MAGISK_v26_APK_FILE="/data/backup/Android/EssentialApps_Backup/Magisk-v27.0.apk" MAGISK_v25_APK_FILE="/data/backup/Android/EssentialApps_Backup/Magisk-v25.2.apk" # default Magisk APK file - this must be Magisk Version 25.x or older! # The script install this Magisk version first; later Magisk is updated with the Magisk version in the variable MAGISK_V26_APK_FILE # MAGISK_APK_FILE="/data/backup/Android/EssentialApps_Backup/Magisk-v25.2.apk" #MAGISK_APK_FILE="/data/backup/Android/EssentialApps_Backup/Magisk-v24.2.apk" # # use "none" or an empty directory to disable installing essential apps # ESSENTIAL_APPS_DIR="/data/backup/Android/EssentialApps" # use "none" or an empty directory to disable installing the scripts # ESSENTIAL_SCRIPTS_DIR="/data/backup/Android/EssentialScripts/" # use "none" or an empty directory to disable install the Magisk Modules # ESSENTIAL_MAGISK_MODULES_DIR="/data/backup/Android/EssentialMagiskModules" # # Magisk Module with MicroG # MICROG_MAGISK_MODULE="/data/backup/Android/MicroG_MagiskModule/microG-GApps.zip" UPLOAD_DIR_FOR_SCRIPTS_ON_THE_PHONE="/sdcard/Download/scripts" UPLOAD_DIR_FOR_MAGISK_MODULES_ON_THE_PHONE="/sdcard/Download/MagiskModules" APPS_FOR_ROOT_ACCESS="com.mixplorer,com.keramidas.TitaniumBackup,io.github.muntashirakon.AppManager,com.matoski.adbm,com.fox2code.mmm" # default TCP port for adb via WiFi # DEFAULT_ADBD_TCP_PORT="5555" # --------------------------------------------------------------------- # Titanium license file # TITANIUM_LICENSE_FILE="/data/backup/Android/Licenses/TitaniumBackup_license.txt" # default Titanium update.zip file to restore the backups # TITANIUM_UPDATE_ZIP_FILE="/data/backup/ASUS_ZENFONE8/current_update.zip" # Titanium files and directories on the phone # TITANIUM_LICENSE_FILE_ON_THE_PHONE="/sdcard/TitaniumBackup_license.txt" # this variable can either contain a directory name ending with a slash "/" or a file name # TITANIUM_UPDATE_ZIP_FILE_ON_THE_PHONE="/sdcard/Download/" # directory with the backups created by Titanium # TITANIUM_BACKUP_DIR_ON_THE_PHONE="/sdcard/TitaniumBackup" # Titanium app # TITANIUM_APP="com.keramidas.TitaniumBackup" # default post install script; scripts without a fully qualified filename are searched in the current directory # POST_INSTALL_SCRIPT="post_install.sh" # default archive with config files for the Android OS # ARCHIVE_FILE="/data/develop/android/android_config.tar" # --------------------------------------------------------------------- # environment variables used by the scripts executed in the tasks # # (remove the leading hash "#" to activate the setting # # --------------------------------------------------------------------- ### Environment variables for the script ./boot_phone_from_twrp.sh #TWRP_IMAGE="" #SERIAL_NUMBER="" #ADB="/usr/bin/adb" #FASTBOOT="/usr/bin/fastboot" #TIMEOUT="/usr/bin/timeout" # These variables are also used by the scripts install_apk.sh, install_magisk_via_twrp.sh, and install_twrp.sh # #ADB_OPTIONS="" #FASTBOOT_OPTIONS="" # --------------------------------------------------------------------- ### Environment variables for the script ./enable_access_via_adb.sh # PUBLIC_KEY_ON_PC="/data/develop/android/adb.pubkey" # --------------------------------------------------------------------- ### Environment variables for the script ./enable_root_access_via_magisk.sh #MAGISK_DATA_DIR="/data/adb" #BIN_DIR="/system/bin" #TMPDIR="/cache/enable_root_access_via_magisk.sh.1115565" #SQLITE3="/usr/bin/sqlite3" # --------------------------------------------------------------------- ### Environment variables for the script ./install_os_via_twrp.sh #UPLOAD_DIR_ON_THE_PHONE="" # --------------------------------------------------------------------- # # Default time out values # # time to wait until the adb daemon is ready after a reboot in seconds # # ADB_DAEMON_WAIT_TIME=30 # time to wait until the phone is booted into fastboot mode in seconds # # FASTBOOT_WAIT_TIME=60 # time to wait until the adb is started after a reboot in seconds # # ADB_BOOT_WAIT_TIME=120 # time to wait until the /data partition is decrypted after starting the decription in seconds # # DECRYPT_DATA_WAIT_TIME=150 # --------------------------------------------------------------------- [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

It's also possible to define other variables used by the various scripts executed by the tasks in prepare_phone.include in this file, an example for the file prepare_phone.conf can be downloaded here.

The config file "prepare_phone.conf" is searched in the current directory, the home directory "${HOME}", and in the directory /etc (in this order),

When done use the task check_config to check the configuration, e.g.

./prepare_phone.sh check_config

The task check_config will check that all required variables are set and, if possible, that the values of these variables are valid. It also checks that all scripts used by the tasks in the include file exist.

The include file defines a task group called all that can be used to install and configure a phone attached via USB from scratch.

To list the tasks that will be done if the parameter all is used use the parameter --list-default-tasks, e.g.

./prepare_phone.sh --list_default_tasks

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ date Wed May 22 01:57:52 PM CEST 2024 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./prepare_phone.sh --list_default_tasks [22.05.2024 13:57 ] ### prepare_phone.sh started at 22.05.2024 13:57:59 (The PID of this process is 161238) [22.05.2024 13:57 ] ### Processing the parameter ... [22.05.2024 13:57 ] Tasks to execute are: "" [22.05.2024 13:57 ] Parameter for the function init_tasks are: "" [22.05.2024 13:57 ] ### The logfile used is /var/tmp/prepare_phone.sh.log [22.05.2024 13:57 ] Checking the include file "/data/develop/android/scripts_on_linux/prepare_phone.include" ... [22.05.2024 13:57 ] Reading the include file "/data/develop/android/scripts_on_linux/prepare_phone.include" ... Include files used are: /data/develop/android/scripts_on_linux/prepare_phone.include Tasks defined are 29 default task(s) defined: enable_disable_abort_on_error:enable select_twrp_image:ignorerc boot_phone_from_twrp_image:force install_os:noreboot:factory_reset restart_adb_daemon backup_boot_partition:next install_twrp:noreboot:next install_magisk_in_boot_partition_only:noreboot:next enable_adb:reboot execute_script_on_the_phone:ignorerc:pre_install.sh wait_for_access_via_adb:240 select_twrp_image:ignorerc install_magisk_in_data_adb_only:noreboot:copy_apk:use_apk wait_for_access_via_adb:120 enable_root_access_for_the_shell:reboot wait_for_access_via_adb:120 install_magisk_v26_in_boot_partition:active:reboot wait_for_access_via_adb:120 install_the_magisk_app install_archive_file:twrp:reboot:script=/cache/correct_setlinux_for_wifconifg.sh enable_disable_abort_on_error:disable install_essential_scripts execute_script_on_the_phone:ignorerc:/data/develop/android/scripts_on_linux/enable_wlan.sh install_essential_apps:ignorerc install_essential_magisk_modules install_magisk_modules:${MICROG_MAGISK_MODULE}:ignorerc enable_root_access_for_apps:reboot execute_script_on_the_phone:ignorerc wait_for_access_via_adb 0 task(s) will be ignored if the parameter "all" is used: [22.05.2024 13:57 ] ### The logfile used was /var/tmp/prepare_phone.sh.log [22.05.2024 13:57 ] ### The start time was 22.05.2024 13:57:59, the script runtime is (day:hour:minute:seconds) 0:00:00:00 (= 0 seconds) [22.05.2024 13:57 ] ### prepare_phone.sh ended at 22.05.2024 13:57:59 (The PID of this process is 161238; the RC is 0) [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Hopefully the task names are self-explanatory ...

In principle the tasks executed if the parameter all is used will

do a factory reset
install the OS using TWRP
backup the initial original boot partition for the new installed OS
install the TWRP recovery into the boot partition
install Magisk v25 into the boot partition
enable access via adb using Magisk
install the Magisk App v25
enable root access for the shell via Magisk
update Magisk to Magisk v27
unpack an archive file with some config files on the phone
copy all scripts from the local directory ${ESSENTIAL_SCRIPTS_DIR} to the directory ${SCRIPT_DIR_ON_THE_PHONE} on the phone
install all apk files found in the local directory ${ESSENTIAL_APPS_DIR}
install all Magisk Modules found in the local directory ${ESSENTIAL_MAGISK_MODULES_DIR}
enable root access for the applications listed in the variable ${APPS_FOR_ROOT_ACCESS}
copy the script post_install.sh from the current directory to the phone and execute it there (only if it exists)

The variables used in the list above can be defined in the file prepare_phone.conf.

To install and configure a phone attached via USB use the command:

./prepare_phone.sh all

Sample output of an installation and configuration done using the script with the parameter all can be found here.

Another log file created while installing and configuring OmniROM 13 on the ASUS Zenfone 8 can be found here; a log file created while installing and configuring OmniROM 14 on the ASUS Zenfone 8 can be found here.

There are also some other task groups defined:

./prepare_phone.sh list_taskgroups

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ date Thu May 2 11:54:20 AM CEST 2024 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./prepare_phone.sh list_taskgroups [02.05.2024 11:54 ] ### prepare_phone.sh started at 02.05.2024 11:54:28 (The PID of this process is 688024) [02.05.2024 11:54 ] ### Processing the parameter ... [02.05.2024 11:54 ] Not yet used parameter are: "list_taskgroups" [02.05.2024 11:54 ] Tasks to execute are: "list_taskgroups" [02.05.2024 11:54 ] Parameter for the function init_tasks are: "" [02.05.2024 11:54 ] ### The logfile used is /var/tmp/prepare_phone.sh.log [02.05.2024 11:54 ] ERROR: Can not write to the file /var/tmp/prepare_phone.sh.log - now using the log file /var/tmp/prepare_phone.sh.log.688024 [02.05.2024 11:54 ] Checking the include file "/data/develop/android/scripts_on_linux/prepare_phone.include" ... [02.05.2024 11:54 ] Reading the include file "/data/develop/android/scripts_on_linux/prepare_phone.include" ... [02.05.2024 11:54 ] Processing the parameter ... [02.05.2024 11:54 ] Preparing the list of tasks to execute ... [02.05.2024 11:54 ] "init_tasks" is defined - now executing it ... ---------------------------------------------------------------------- [02.05.2024 11:54 ] Searching for a config file .... [02.05.2024 11:54 ] Config file "prepare_phone.conf" succesfully read [02.05.2024 11:54 ] Using the directory "/tmp" for temporary files [02.05.2024 11:54 ] The PID of this process is 688024 [02.05.2024 11:54 ] The include file used is : -rw-r--r--. 1 xtrnaw7 xtrnaw7 467983 May 2 10:40 /data/develop/android/scripts_on_linux/prepare_phone.include [02.05.2024 11:54 ] The include file used was last edited at: "30.04.2024 /bs" [02.05.2024 11:54 ] The version of the used execute_tasks.sh script is "v2.2.5" ----------------------------------------------------------------------- [02.05.2024 11:54 ] Using the TWRP image "/data/backup/ASUS_ZENFONE8/twrp/current_twrp.img" for booting the phone into TWRP [02.05.2024 11:54 ] Using the working directory "/tmp/prepare_phone.sh_WORKDIR.688024" . [02.05.2024 11:54 ] *** Reading the include file "/data/develop/android/scripts_on_linux/boot_phone_from_twrp.sh" ... [02.05.2024 11:54 ] The script is running as user "xtrnaw7" -- will use "sudo" for the fastboot commands ... [02.05.2024 11:54 ] Will use the attached phone with the serial number "M6AIB760D0939LX" [02.05.2024 11:54 ] Using the TWRP image defined in the environment variable TWRP_IMAGE: "/data/backup/ASUS_ZENFONE8/twrp/current_twrp.img" [02.05.2024 11:54 ] Using the options "-d -s M6AIB760D0939LX " for the adb commands [02.05.2024 11:54 ] Using the options " -s M6AIB760D0939LX " for the fastboot commands [02.05.2024 11:54 ] Checking the script prerequisites ... [02.05.2024 11:54 ] Will wait up to 60 second(s) after booting the phone into the bootloader [02.05.2024 11:54 ] Will wait up to 120 second(s) after booting the phone from the TWRP image [02.05.2024 11:54 ] Will wait up to 30 second(s) until the adb daemon is ready to use [02.05.2024 11:54 ] The phone is currently booted into the Android OS [02.05.2024 11:54 ] Selecting the TWRP image to use depending on the installed OS on the phone ... [02.05.2024 11:54 ] Checking the installed OS on the phone ... [02.05.2024 11:54 ] The OS on the phone is LineageOS 20.0 [02.05.2024 11:54 ] The TWRP image for LineageOS 20 is "/data/backup/ASUS_ZENFONE8/Lineage-20/twrp_3.7.0_12-0-I006D_for_lineageOS20-20240423-sake.img" [02.05.2024 11:54 ] Checking the type of the file "/data1/backup/ASUS_ZENFONE8/Lineage-20/twrp_3.7.0_12-0-I006D_for_lineageOS20-20240423-sake.img" ... [02.05.2024 11:54 ] The file type is "Android bootimg, kernel (0x2a1eb26), ramdisk (0x630)" [02.05.2024 11:54 ] OK, the file "/data1/backup/ASUS_ZENFONE8/Lineage-20/twrp_3.7.0_12-0-I006D_for_lineageOS20-20240423-sake.img" is a valid boot image for Android devices [02.05.2024 11:54 ] The installed OS is "LineageOS" [02.05.2024 11:54 ] The TWRP file necessary for this OS is "/data1/backup/ASUS_ZENFONE8/Lineage-20/twrp_3.7.0_12-0-I006D_for_lineageOS20-20240423-sake.img" [02.05.2024 11:54 ] The boot slot is _b. ---------------------------------------------------------------------- [02.05.2024 11:54 ] Executing the tasks now ... [02.05.2024 11:54 ] The tasks to execute are: Task 1: list_taskgroups [02.05.2024 11:54 ] Processing the task "list_taskgroups" ... [02.05.2024 11:54 ] Executing now "list_taskgroups " ---------------------------------------------------------------------- [02.05.2024 11:54 ] The defined task groups are: all - install the OS into the inactive slot, Customize the OS and install Applications, Scripts, Magisk, Magisk Modules, TWRP all_test - task group for testing only all_sideload - install the OS via sideload into the inactive slot, customize the OS, and install Applications, Scripts, Magisk, Magisk Modules, and TWRP all_no_magisk_no_twrp - install the OS into the inactive slot, enable adb, install Applications; do not install TWRP and deinstall Magisk after the customizing is done reset_and_install_os - install the OS into the inactive slot via install or sideload (auto) and do a factory reset all_without_os - all default tasks w/o installing the OS into the inactive slot os_twrp_magisk - install the OS into the inactive slot, and install TWRP and Magisk update_os_w_Magisk25 - update the OS; install TWRP and Magisk v25 into the boot partition for the new OS update_os - install TWRP and Magisk v26 (or newer) into the boot partition for the new OS install_magisk - Install Magisk raw_image_with_magisk - install the raw Android image from ASUS, install Magisk, enable adb // Test Entry! abort_on_error - abort the task execution after a task ends with an error no_abort_on_error - do not abort the task execution after a task ends with an error wait_for_bootloader - wait until the phone is booted into the bootloader wait_for_fastboot - wait until the phone is booted into the fastbootd wait_for_androidos - wait until the phone is booted into the Android OS wait_for_safemode - wait until the phone is booted into the Android OS in safe mode wait_for_twrp_image - wait until the phone is booted from the TWRP image wait_for_twrp_recovery - wait until the phone is booted from the TWRP recovery wait_for_lineageos_recovery - wait until the phone is booted into the recovery from the LineageOS wait_for_sideload - wait until the phone is booted into the sideload mode refresh - download the current version of the file prepare_phone.include refresh_all - download the current version of all used scripts and the file prepare_phone.include ---------------------------------------------------------------------- [02.05.2024 11:54 ] "finish_tasks" is defined - now executing it ... ---------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------- [02.05.2024 11:54 ] *** Task summary : ------------------------------------------------------------------------------------------------------------- [02.05.2024 11:54 ] Tasks executed successfully are: list_taskgroups ------------------------------------------------------------------------------------------------------------- [02.05.2024 11:54 ] The working directory used is: "/tmp/prepare_phone.sh_WORKDIR.688024" . [02.05.2024 11:54 ] The include file used was : [02.05.2024 11:54 ] -rw-r--r--. 1 xtrnaw7 xtrnaw7 467983 May 2 10:40 /data/develop/android/scripts_on_linux/prepare_phone.include [02.05.2024 11:54 ] The include file used was last edited at: "30.04.2024 /bs" ---------------------------------------------------------------------- [02.05.2024 11:54 ] Execution summary: [02.05.2024 11:54 ] Tasks processed: 1 [02.05.2024 11:54 ] Tasks executed succesfully: 1 [02.05.2024 11:54 ] Tasks executed with errors: 0 [02.05.2024 11:54 ] Tasks not executed on request: 0 [02.05.2024 11:54 ] Tasks not found: 0 [02.05.2024 11:54 ] ### The logfile used was /var/tmp/prepare_phone.sh.log.688024 [02.05.2024 11:54 ] ### The start time was 02.05.2024 11:54:28, the script runtime is (day:hour:minute:seconds) 0:00:00:01 (= 1 seconds) [02.05.2024 11:54 ] ### prepare_phone.sh ended at 02.05.2024 11:54:29 (The PID of this process is 688024; the RC is 0) [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

The task group update_os can be used together with the task set_os_image_to_install to update an existing OS, e.g:

./prepare_phone.sh set_os_image_to_install:/data/develop/android/OmniROM_13.0/out/target/product/zenfone8/omni-13-20230121-zenfone8-MICROG.zip update_os

Sample output of an OS update using these tasks can be found here.

To install and configure the phone manually using my other scripts use these commands:

Manual installation and configuration of the phone

export OS_IMAGE_TO_INSTALL="/data/develop/android/OmniROM_13.0/out/target/product/zenfone8/omni-13-20230115-zenfone8-MICROG.zip" export TWRP_IMAGE="/data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D-enhanced.img" export MAGISK_APK_FILE="/data/backup/Android/EssentialApps/Magisk-v25.2.apk" cd /data/develop/android/scripts_on_linux ./boot_phone_from_twrp.sh ./install_os_via_twrp.sh factory_reset noreboot $OS_IMAGE_TO_INSTALL ./install_twrp.sh next noreboot ./install_magisk_via_twrp.sh next copy_apk delete_adb_dir noreboot magisk_apk_file=$MAGISK_APK_FILE ./enable_access_via_adb.sh reboot adb wait-for-device ./install_magisk_via_twrp.sh adb_only copy_apk noreboot ./init_magisk_db.sh reboot ./install_apk.sh $MAGISK_APK_FILE adb shell pm grant com.topjohnwu.magisk android.permission.POST_NOTIFICATIONS # to install an app (example) ./install_apk.sh /data/backup/Android/EssentialApps/MiXplorer_v6.58.8_B23010320.apk # to install a Magisk Module (example) adb push /data/backup/Android/EssentialMagiskModules/addbin-v1.3.0.0.zip /sdcard/Download/MagiskModules/addbin-v1.3.0.0.zip adb shell su - -c magisk --install-module /sdcard/Download/MagiskModules/addbin-v1.3.0.0.zip # to enable root access for other apps (example) # ./init_magisk_db.sh apps=+com.mixplorer,com.keramidas.TitaniumBackup,io.github.muntashirakon.AppManager,com.matoski.adbm,com.fox2code.mmm

To update the OS manual using the scripts use these commands (assuming Magisk is already installed):

Manual update of the OS on the phone

export OS_IMAGE_TO_INSTALL="/data/develop/android/OmniROM_13.0/out/target/product/zenfone8/omni-13-20230115-zenfone8-MICROG.zip" export TWRP_IMAGE="/data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D-enhanced.img" cd /data/develop/android/scripts_on_linux ./boot_phone_from_twrp.sh ./install_os_via_twrp.sh noreboot $OS_IMAGE_TO_INSTALL ./install_twrp.sh next noreboot ./install_magisk_via_twrp.sh next reboot

Notes:

Reinstalling an ASUS Zenfone 8 with OmniROM from scratch including a factory reset takes about 22 minutes with these scripts, and updating the operating system in the passive slot takes about 5 minutes.

Please note that while writing the global script I updated and corrected some of the other scripts. Therefor it's recommended to download the current version of the scripts.

Some hints for trouble shooting

Each task is a function in the include file with the name task_<taskname>, e.g. the function used to implement the task enable_adb is task_enable_adb.

Task parameter must be added to the taskname using the colon ":" as delimiter, example: install_magisk_in_boot_partition_only:noreboot:next

To execute the tasks in verbose mode use the parameter --info; to execute the complete script in verbose mode use the parameter --verbose.

To execute the tasks with "set -x" use the parameter --trace.

To execute the tasks in single step mode one after each other. use the parameter --singlestep.

The global initialisation of the script is done in the function init_tasks; all script parameter after "--" are parameter for the function init_tasks. To execute the function init_tasks with "set -x" use the parameter "-- trace"; to view the known parameter for the function init_tasks use the parameter "-- help".

To execute the tasks in dry-run mode use the init_tasks parameter " -- dryrun"

In the default configuration execute_tasks.sh will stop executing the remaining tasks if one of the tasks ends with an error. To disable this functionality for one task use the task parameter ignorerc; to disable this functionality global use the the task
enable_disable_abort_on_error:disable and to enable it again use the task enable_disable_abort_on_error:enable.

Use the parameter "-h -v" to get the detailed usage help

Notes

The scripts used by this script can be downloaded manually from these URLs:

https://bnsmb.de/files/public/Android/boot_phone_from_twrp.sh
https://bnsmb.de/files/public/Android/enable_access_via_adb.sh
https://bnsmb.de/files/public/Android/enable_adb_using_magisk.sh
https://bnsmb.de/files/public/Android/enable_root_access_via_magisk.sh
https://bnsmb.de/files/public/Android/init_magisk_db.sh
https://bnsmb.de/files/public/Android/install_apk.sh
https://bnsmb.de/files/public/Android/install_magisk_via_twrp.sh
https://bnsmb.de/files/public/Android/install_os_via_twrp.sh
https://bnsmb.de/files/public/Android/install_twrp_from_within_twrp.sh
https://bnsmb.de/files/public/Android/install_twrp.sh

In addition, the executable usbreset is now used to reset the USB port (see How to reset the USB port used to connect an Android phone ):

https://bnsmb.de/files/public/Android/usbreset

The documentation for these scripts can be found on my home page https://bnsmb.de/My_HowTos_for_Android.html or in the various HowTos in the XDA Forum that I wrote.

History of this entry

Update 02.02.2023 /bs

See here for how to configure WiFi in Android via script

Update 18.04.2023 /bs

see How to install Magisk v26.0 or newer via script for how to install Magisk v26 via script

Update 10.10.2023 /bs

see the entry How to install the Android OS and Magisk via script and also enable access via adb for an example for using the script

Update 12.01.2024 /bs

In TWRP version 3.7.0_12-1 for the ASUS Zenfone 8 (necessary for Android 14 based ROMs) the bootctl binary is not included anymore. Therefor the installation and configuration described in this post will only partially work.
To work around this, you need to create a new TWRP image based on TWRP 3.7.0_12-1 but with the bootctl binary (see here for instructions how to do this)

Update 13.01.2024 /bs

see How to compile the image for TWRP 3.7.0.12-1 w/ the bootctl binary for how to compile a image for TWRP 3.7.0.12-1 that contains the bootctl binary

Update 25.04.2024 /bs

Updated the list of tasks and task groups to match the current version of the script; new task and task groups are;

Update details 01.01.2024 - 25.04.204

#H# 19.01.2024 /bs #H# all tasks now print the task usage if an invalid task parameter was used #H# added the task task_restart_adbd_on_the_phone #H# added the task task_enable_adb_via_wifi #H# added the task task_connect_using_adb_via_wifi #H# #H# 20.01.2024 /bs #H# corrected an "error" in the task task_connect_using_adb_via_wifi : adb returns 0 even if the command fails #H# #H# 01.04.2024 /bs #H# added the task task_list_taskgroups #H# #H# 02.04.2024 /bs #H# the task task_enable_adb ignored the contents of the environment variable PUBLIC_KEY_ON_PC -- fixed #H# the script now exports all variables defined in the file prepare_phone.conf by default #H# #H# 25.04.2024 /bs #H# added the task task_backup_boot_partition #H# added the task task_restore_boot_partition #H# added the task task_set_twrp_image_to_use #H# added the task task_select_twrp_image #H# added the task task_execute_command_on_the_phone #H# installing the OS via default tasks now creates a backup of the boot partition in the file #H# /data/recovery/initial_boot${BOOT_SLOT}.img #H# ${BOOT_SLOT} is the slot with the new installed OS: either "_a" or "_b" #H# the task group all_no_magisk_no_twrp can be used to install the OS and all Apps with access via adb but without Magisk and TWRP #H#

Update 03.05.2024 /bs

Updated the list of tasks and task groups to match the current version of the script; new task and task groups are;

Update details 26.04.2024 - 03.05.204

#H# 27.04.2024 /bs #H# added support for ROMs with plain root access (e.g no "su -" required to become root) #H# fixed some errors in the messages #H# rewrote some code in the function task_check_config #H# #H# 29.04.2024 /bs #H# added the task task_install_ASUS_raw_image #H# #H# 03.05.2024 /bs #H# added code to change the TWRP image to use after a script executed with execute_script failed #H# the task task_enable_adb now supports the parameter to select the slot #H# there is now a separate task to install the Magisk Module with MicroG (that module can only be used in the OmniROM) #H# added the task task_restart_adb_daemon #H# the init task parameter "nophonecheck" only worked partially -- fixed #H#

Update 22.05.2024 /bs

Updated the list of task, the contents of the file prepare_phone.conf to match the current version of the scripts

Update details 04.05.2024 - 18.05.204

#H# 04.05.2024 /bs #H# added the task task_select_twrp_image_for_install_image #H# added the statement #H# ENABLE_SINGLE_STEP_ON_ERROR=${__TRUE} #H# (-> prepare_phone.sh will now enable single step mode if a task fails) #H# #H# 05.05.2024 /bs #H# the task install_os now restarts the adb daemon as user root using sudo if it's running as non-root user and installing via sideload is requested #H# TWRP will not be installed in the boot partition any more if the OS image is installed via sideload #H# task task_select_twrp_image_for_install_image : added support for /e/ #H# task task_select_twrp_image : added support for /e/ (in the function get_twrp_image_for_the_installed_OS in boot_phone_from_twrp.sh) #H# task task_enable_adb ended with RC=${__TRUE} even if the final reboot failed -- fixed #H# #H# 07.05.2024 /bs #H# added the task task_DebugShell #H# #H# 10.05.2024 /bs #H# added the task task_copy_file_to_phone #H# #H# 18.05.2024 /bs #H# added the task task_reset_usb_port #H# added the task task_enable_single_step_mode #H#

-----

How to install and configure other ROMs via script

URL: not yet published, but the entry is referenced in this post: https://xdaforums.com/t/how-to-install-and-configure-the-android-os-via-script.4546375/#post-89489846

How to install and configure other ROMs via script

The instructions to install the Android OS on a phone using the script prepare_phone.sh described in the entry How to install and configure the Android OS via Script work only for ROMs that can be installed using the TWRP binary twrp with the parameter install. For the ASUS Zenfone 8 that is only the OmniROM as far as I know.

Now that the script install_os_via_twrp.sh also supports installing an OS using the parameter sideload of the binary twrp (see here for details) it's also possible to install the original Android from ASUS, the LineageOS or any other AOSP based OS like StatiXOS, LMODroid, or /e/.

(For details see How to install an OS image via sideload using the TWRP binary twrp.)

To install these operating systems via script, I have added some tasks and task groups to the include file prepare_phone.include used by the script prepare_phone.sh to install and configure the Android OS automatically.

Please note that changes to the other scripts used by prepare_phone.sh and config file prepare_phone.conf were also necessary for these new tasks. So if you want to use the new tasks in prepare_phone.include for the script prepare_phone.sh, please download the latest version of the scripts first, either manually from /files/public/Android or via the prepare_phone.sh script using this task:

./prepare_phone.sh refresh_all

After updating the scripts as described above the original Android from ASUS can be installed using these tasks:

Define the installation image for the original Android from ASUS in the config file prepare_phone.conf, e.g:
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ grep "^OS_IMAGE_TO_INSTALL" prepare_phone.conf OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/ASUS_firmware/Android_13/UL-ASUS_I006D-ASUS-33.0210.0210.296-1.1.26-2305-user.zip" [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Then install and configure the OS using this command

./prepare_phone.sh all_sideload

The tasks in the task group all_sideload are:

task group all_sideload

# # History # 04.05.2024 # removed the task install_twrp # TASK_GROUP_all_sideload="# enable_disable_abort_on_error:enable # select the correct TWRP image for the running OS on the phone # select_twrp_image:ignorerc # # install the OS # boot_phone_from_twrp_image # install the OS image via sideload # (if using auto here the following tasks will fail because the necessary boot from the TWRP image after the installation changes the current boot slot) # install_os:noreboot:sideload:factory_reset:restart_as_root # select the TWRP image necessary for the just installed OS (define the environment variable NEW_TWRP_IMAGE to disable this task) # select_twrp_image_for_install_image # # after install an OS image using the sideload feature the /data partition is not mounted -> reboot from the TWRP image to get a mounted /data partition for the next tasks # boot_phone_from_twrp_image:force # IMPORTANT: now all tasks must be done in the active boot slot! # create a backup of the initial boot partition in /data/recovery/initial_boot${SLOT_FOR_THIS_TASK}.img # backup_boot_partition:active # install TWRP in the boot partition # # install_twrp:noreboot:active # # first install Magisk v25.x into the boot partition # install_magisk_in_boot_partition_only:noreboot:active enable_adb:reboot:active # do first configurations via Android cli commands if necessary # execute_script_on_the_phone:ignorerc:pre_install.sh # select the correct TWRP image for the running OS on the phone # select_twrp_image:ignorerc # install_magisk_in_data_adb_only:noreboot:copy_apk:use_apk wait_for_access_via_adb:120:kill enable_root_access_for_the_shell:reboot wait_for_access_via_adb:120:kill # # the next step is necessary to install Magisk v26 or newer # install_magisk_v26_in_boot_partition:active:reboot install_the_magisk_app wait_for_access_via_adb:120:kill install_archive_file:twrp:reboot:script=/cache/correct_setlinux_for_wifconifg.sh enable_disable_abort_on_error:disable install_essential_scripts execute_script_on_the_phone:ignorerc:/data/develop/android/scripts_on_linux/enable_wlan.sh # use a separate directory for the apps to install for non-OmniROM ROMs # install_essential_apps:/data/backup/Android/Essential_Apps_for_ASUS_Android install_essential_magisk_modules:ignorerc enable_root_access_for_apps:reboot execute_script_on_the_phone:ignorerc "

Examples

Logfile of the installation and customization of the original ASUS Android 13 using the script prepare_phone.sh

To install the LineageOS (either version 20 or version 21) or an LineageOS based OS an additional step is necessary:

The automatic installation of the Android OS requires a working TWRP image. As 01.05.2024 the official TWRP image (at least the TWRP image for the ASUS Zenfone 8) does still not boot if the LineageOS 20 or newer is installed on the phone. Therfor to install the LineageOS 20+, you must first create a TWRP image for the LineageOS version you want to install. That's not that difficult and can be done quickly in any Linux OS or in an adb shell on the phone - see the entry How to use TWRP if LineageOS 20.x is installed to learn how to do this.

After creating the necessary TWRP image for the LineageOS do

1. Define the LineageOS image to be installed in the file prepare_phone.conf, e.g. for the LineageOS 20:
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ grep "^OS_IMAGE_TO_INSTALL=" prepare_phone.conf OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/Lineage-20/lineage-20.0-20240423-nightly-sake-signed.zip" [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

2. Define the TWRP image to be used for installing and configuring the LineageOS in the variables

TWRP_IMAGES_FOR_IMAGE_FILES

and

TWRP_IMAGES_FOR_THE_RUNNING_OS

in the file prepare_phone.conf:

Example

# ------------------ # TWRP images for specific ROM image files # # each line must contain these fields (the field separtor is ":") : # # regex for the image file : TWRP image to use : Name of the ROM : description # # The fields for the name of the ROM and the description are optional # TWRP_IMAGES_FOR_IMAGE_FILES=" # UL-ASUS* : ${DEFAULT_TWRP_IMAGE} : ASUS Android LMODroid* : /data/backup/ASUS_ZENFONE8/LMODroid/twrp_LMODroid-4.2-20240429-RELEASE-sake.img : LMODroid : e-1.21* : /data1/backup/ASUS_ZENFONE8/e/twrp_recovery-e-1.21-t-20240325389105-dev-sake.img : /e/ lineage-20* : /data/backup/ASUS_ZENFONE8/Lineage-20/twrp_3.7.0_12-1-I006D_for_lineageOS20-20240423-sake.img : LineageOS 20.x lineage-21* : /data/backup/ASUS_ZENFONE8/Lineage-21/twrp_3.7.0_12-1-I006D_for_lineageOS21-20240220-sake.img : LineageOS 21.x statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE.zip : /data/backup/ASUS_ZENFONE8/Statix/20240106/twrp_statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE.img : StatixOS statix_sake-20231224-14-v7.1-UPSIDEDOWNCAKE.zip : /data/backup/ASUS_ZENFONE8/Statix/20231229/statix_sake-20231224_twrp.img : StatixOS # " # ------------------ # TWRP images for specific ROMs currently running on the phone # # each line must contain these fields (the field separtor is ":") : # # property : value : TWRP image to use : Name of the ROM : description # # use value = "*" if the property must be defined with any value # # The fields for the name of the ROM and the description are optional # # TWRP_IMAGES_FOR_THE_RUNNING_OS=" # vendor.asus.build.ext.version : * : ${DEFAULT_TWRP_IMAGE} : ASUS Android ro.lineage.build.version : 20* : /data/backup/ASUS_ZENFONE8/Lineage-20/twrp_3.7.0_12-1-I006D_for_lineageOS20-20240423-sake.img : LineageOS ro.lineage.build.version : 21* : /data/backup/ASUS_ZENFONE8/Lineage-21/twrp_3.7.0_12-1-I006D_for_lineageOS21-20240220-sake.img : LineageOS ro.statix.version : * : /data/backup/ASUS_ZENFONE8/Statix/20240106/twrp_statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE.img : StatixOS ro.omni.version : * : ${DEFAULT_TWRP_IMAGE} : OmniROM ro.product.system.name : lmodroid_sake : /data/backup/ASUS_ZENFONE8/LMODroid/twrp_LMODroid-4.2-20240429-RELEASE-sake.img : LMODroid "

3. If the phone is NOT booted into the Android OS with working access via adb:

Boot the phone from the TWRP image usable for the current installed OS on the phone

4. Install the LineageOS via sideload

./prepare_phone.sh all_sideload

Notes:

prepare_phone.sh called with the parameter all_sideload restarts the adb daemon as user root using sudo if it's not started yet by the user root.

This approach also works for other LineageOS based ROMs like for example /e/.

It also works for other ASOP based ROMs, like StatiXOS or LMODroid.

Note that for unknown reason the connection between the PC and the phone via fastboot or adb sometimes does not work automatically after installing the OS via sideload. In this case just pull and re-plug the USB cable.

Examples:

Logfile of the installation and customization of LineageOS 20 using the script prepare_phone.sh

Logfile of the installation and customization of LineageOS 21 using the script prepare_phone.s h

Logfile of the installation and customization of /e/ 1.21 using the script prepare_phone.sh

Logfile of the installation and customization of StatiXOS using the script prepare_phone.sh

Logfile of the installation and customization of LMODroid using the script prepare_phone.sh

Other new features in prepare_phone.include

As different TWRP images are required for different ROMs, there are now two tasks that decide which TWRP image should be used.

The task

select_twrp_image

selects the TWRP image to be used depending on the OS currently running on the phone.

The task uses the contents of the variable TWRP_IMAGES_FOR_THE_RUNNING_OS to decide which TWRP image should be used. The variable TWRP_IMAGES_FOR_THE_RUNNING_OS is defined in the file prepare_phone.conf:

variable used by the task select_twrp_image

#TWRP images for specific ROMs currently running on the phone # # each line must contain these fields (the field separtor is the colon ":";lines starting with a hash "#" are ignored) : # # property : value : TWRP image to use : Name of the ROM : description # # Use "*" for the field "value" if the property must be defined but the value is meaningless. # # The fields for the name of the ROM and the description are optional # # TWRP_IMAGES_FOR_THE_RUNNING_OS=" # vendor.asus.build.ext.version : * : ${DEFAULT_TWRP_IMAGE} : ASUS Android ro.lineage.build.version : 20* : /data/backup/ASUS_ZENFONE8/Lineage-20/twrp_3.7.0_12-1-I006D_for_lineageOS20-20240423-sake.img : LineageOS ro.lineage.build.version : 21* : /data/backup/ASUS_ZENFONE8/Lineage-21/twrp_3.7.0_12-1-I006D_for_lineageOS21-20240220-sake.img : LineageOS ro.statix.version : * : /data/backup/ASUS_ZENFONE8/Statix/20240106/twrp_statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE.img : StatixOS ro.product.system.name : lmodroid_sakeX : /data/backup/ASUS_ZENFONE8/LMODroid/twrp_LMODroid-4.2-20240429-RELEASE-sake.img : LMODroid ro.omni.version : * : ${DEFAULT_TWRP_IMAGE} : OmniROM ro.build.description : e_sake* : /data/backup/ASUS_ZENFONE8/e/twrp_recovery-e-1.21-t-20240325389105-dev-sake.img : /e/ "

Note that the task select_twrp_image only works if the phone is booted into the Android OS with a working adb connection (see the task group definitions in the file prepare_phone.include how to use the task).

The task

task_select_twrp_image_for_install_image

selects the TWRP image to be used depending on the name of an OS image file.

The task uses the contents of the variable TWRP_IMAGES_FOR_IMAGE_FILES to decide which TWRP image should be used. The variable TWRP_IMAGES_FOR_IMAGE_FILES is defined in the file prepare_phone.conf:

variable used by the task select_twrp_image_for_install_image

# ------------------ # TWRP images for specific ROM image files # # each line must contain these fields (the field separtor is the colon ":"; lines starting with a hash "#" are ignored) : # # regex for the image file : TWRP image to use : Name of the ROM : description # # The fields for the name of the ROM and the description are optional # TWRP_IMAGES_FOR_IMAGE_FILES=" # UL-ASUS* : ${DEFAULT_TWRP_IMAGE} : ASUS Android LMODroid* : /data/backup/ASUS_ZENFONE8/LMODroid/twrp_LMODroid-4.2-20240429-RELEASE-sake.img : LMODroid : e-1.21* : /data1/backup/ASUS_ZENFONE8/e/twrp_recovery-e-1.21-t-20240325389105-dev-sake.img : /e/ lineage-20* : /data/backup/ASUS_ZENFONE8/Lineage-20/twrp_3.7.0_12-1-I006D_for_lineageOS20-20240423-sake.img : LineageOS 20.x lineage-21* : /data/backup/ASUS_ZENFONE8/Lineage-21/twrp_3.7.0_12-1-I006D_for_lineageOS21-20240220-sake.img : LineageOS 21.x statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE.zip : /data/backup/ASUS_ZENFONE8/Statix/20240106/twrp_statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE.img : StatixOS statix_sake-20231224-14-v7.1-UPSIDEDOWNCAKE.zip : /data/backup/ASUS_ZENFONE8/Statix/20231229/statix_sake-20231224_twrp.img : StatixOS omni* : ${DEFAULT_TWRP_IMAGE} : OmniROM # "

As of 06.05.2024 both tasks detects these ROMs:

ASUS Android OS
OmniROM
LineageOS
StatiXOS
/e/
LMODroid

There is now a task to backup the boot partition and another task to restore the boot partition:

backup_boot_partition
restore_boot_partition

With these tasks it is possible to install and configure the operating system on the phone and to remove the Magisk files required for configuring the operating system after the configuration of the phone is done. To use this functionality the task group all_no_magisk_no_twrp can be used.

The tasks in the task group all_no_magisk_no_twrp are:

task group all_no_magisk_no_twrp

TASK_GROUP_all_no_magisk_no_twrp="# enable_disable_abort_on_error:enable # select the correct TWRP image for the running OS on the phone # select_twrp_image:ignorerc # # install the OS # boot_phone_from_twrp_image # Note: Task parameter 'auto' -> select the installation method ('twrp install' or 'twrp sideload') depending on the file name of the OS image to install # use 'sideload' instead of auto to force installing via 'twrp sideload' # install_os:noreboot:factory_reset:auto # the adb daemon sometimes hangs after installing the new image # kill_adb_daemon # create a backup of the initial boot partition in /data/recovery/initial_boot${SLOT_FOR_THIS_TASK}.img # backup_boot_partition:next # # first install Magisk v25.x into the boot partition; this installation is necessary to # enable the access via adb to the phone; the Magisk installation will be overwritten later # with Magisk v26 or newer # install_magisk_in_boot_partition_only:noreboot:next enable_adb:reboot # do first configurations via Android cli commands if necessary # execute_script_on_the_phone:ignorerc:pre_install.sh # wait until the new OS is up and running with working adb access # wait_for_access_via_adb:240:kill # select the correct TWRP image for the running OS on the phone # select_twrp_image:ignorerc # # install Magisk in the boot partition # install_magisk_in_data_adb_only:noreboot:copy_apk:use_apk wait_for_access_via_adb:120:kill # enable adb # enable_root_access_for_the_shell:reboot wait_for_access_via_adb:120:kill install_archive_file:twrp:reboot:script=/cache/correct_setlinux_for_wifconifg.sh enable_disable_abort_on_error:disable execute_script_on_the_phone:ignorerc:/data/develop/android/scripts_on_linux/enable_wlan.sh # Access via adb is working now and we do not need root access anymore -- we do not need Magisk anymore # restore_boot_partition:twrp:noreboot install_essential_apps execute_script_on_the_phone:ignorerc "

The usage for the two tasks is

usage: task_backup_boot_partition [slot] [reboot|reboot=no|reboot=yes] [ignorerc] [twrp] [notwrp] [twrp=twrp_image] [[phone=]image_file_on_the_phone] [[pc=]image_file_on_the_pc] usage: task_restore_boot_partition [slot] [reboot|reboot=no|reboot=yes] [ignorerc] [twrp] [twrp=twrp_image] [[pc=|phone=]image_file]
So it's also possible to create multiple backups of both boot partitions in different files (see the source code for the functions task_backup_boot_partition and task_restore_boot_partition in the file prepare_phone.include for further details)

Examples

Logfile of the installation of OmniROM with only temporary installing Magisk using the script prepare_phone.sh

There is now also a task to install one of the raw images with the Android OS on the ASUS Zenfone 8:

install_ASUS_raw_image

The task uses the variable DEFAULT_CUR_RAW_IMAGE_DIR to select the directory with the raw Android image that should be installed:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ grep -i raw prepare_phone.conf # default raw image to be installed DEFAULT_CUR_RAW_IMAGE_DIR="/data/backup/ASUS_ZENFONE8/raw_images/Android13/OPEN-ZS590KS-32.0501.0403.4-BSP-2206-user-20220705" [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $
The installation is done using the script update_image.sh from the raw image.

Using this task it is now also possible to automatically install the raw Android image from ASUS with enabled access via adb using Mgisk. The tasks necessary for this are in the task group raw_image_with_magisk.

The tasks in the task group raw_image_with_magisk are:

task group raw_image_with_magisk

TASK_GROUP_raw_image_with_magisk="# # install_ASUS_raw_image # # /data is not mounted after installing the raw image # -> use TWRP to format /data # boot_phone_from_twrp_image:ignorerc format_data_via_twrp # # reboot the phone from the TWRP image to ensure that the now formated /data partition is mounted # boot_phone_from_twrp_image:force install_magisk_in_boot_partition_only:noreboot:_a enable_adb:reboot:_a install_magisk_in_data_adb_only:noreboot:copy_apk:use_apk wait_for_access_via_adb:120:kill enable_root_access_for_the_shell:reboot wait_for_access_via_adb:120:kill install_the_magisk_app "

To install another ASUS raw image add the name of the directory with the ASUS raw image as task parameter, e.g. :

./prepare_phone.sh install_ASUS_raw_image:/data/backup/ASUS_ZENFONE8/raw_images/Android13/OPEN-ZS590KS-32.0301.0403.10-BSP-2204-user-20220505

Examples

Logfile of the installation of the Android raw image w/ Magisk using the script prepare_phone.sh

Note:

The installation of a raw Android OS image on the ASUS Zenfone 8 is required if the phone is somehow "bricked" and nothing else works anymore. The plain raw Android OS image can also be installed on phones with locked boot loader.
Installing the raw Android OS image will always delete all data on the phone!

-----

History of this entry

03.05.2024

initial releaes

05.05.2024

added infos about new task
added infos about the installation of /e/

07.05.2024

added the logfile for the installation StatiXOS via prepare_phone.sh
added the logfile for the installation LMODroid via prepare_phone.sh

---

How to install the Android OS and Magisk via script and also enable access via adb

URL: https://xdaforums.com/t/how-to-enable-access-via-adb-on-a-new-installed-os.4535165/#post-89088101

How to install the Android OS and Magisk via script and also enable access via adb

In this post I describe how to install the Android OS and Magisk and enable access via adb using the script documented in this forum entry.

As far I understand your log you want to

- install the OS
- install Magisk
- enabe access via adb

(there is no task to install TWRP in the logs)

This can be done with the script with a TWRP version that can handle encrypted /data partitions (TWRP version 3.6 or newer if I remember correctly)
Please note that this installation method for Magisk is only valid for Magisk up to version 25.x. From Magisk version 26.x on a different installation method is necessary
(see https://xdaforums.com/t/how-to-install-magisk-into-the-boot-partition-using-a-script.4456621/#post-88429939)

The meaning of the error message

The reported problem is: "--reason=set_policy_failed:/data/adb"
is:

In Android 13 (probably also in 12, but I'm not sure), Android always uses encryption for the files in the /data partition. The initial key used for encryption is generated by the OS during the first reboot after OS installation. The encryption key is stored in one of the phone's partitions and is required to access the files and directories in /data.

A prerequisite for this approach is that there are no files in the /data partition at the first reboot (except for some subdirectories in /data, which are not encrypted at all).

If the Android operating system finds unencrypted files on the /data partition before the initial encryption was done, it fails with an error message and requests a factory reset: The factory reset will then delete all files on the /data partition, and the next reboot will succeed.

So, if you install and customize the Android operating system via scripts, it is mandatory not to write any files or directories to /data before the first reboot of the newly installed operating system.

The necessary task sequence to do what you want is:

# This task is optional ; the task install_os will reboot the phone from the TWRP image if it's not already booted from the TWRP image # ./prepare_phone.sh boot_phone_from_twrp_image force # install the new OS # ./prepare_phone.sh install_os no:reboot:factory_reset:/data/backup/Redmi_9T/lineage-18.1-20210514-UNOFFICIAL-juice.zip # install Magisk only in the boot partition -- do not store any files in /data # ./prepare_phone.sh install_magisk_in_boot_partition_only:noreboot:next # add the init scripts to enable adb to the ramdisk in the boot partition -- do not store any files in /data # # until now nothing was written to the /data partition therefor the reboot into the new OS will succeed # ./prepare_phone.sh enable_adb:reboot # now the phone booted the first time from the new OS (task parameter "reboot") and the OS initialized the encryption for the /data partition # Therefor we can now write files to /data (either while booted into the OS or while booted into an TWRP image with support for accessing encrypted /data partitions) # ./prepare_phone.sh install_magisk_in_data_adb_only:noreboot:copy_apk:use_apk # optional: enable root access via adb # ./prepare_phone.sh enable_root_access_for_the_shell:reboot # Note that the last task executed before the task to install the Magisk App must reboot the phone # into the Android OS # ./prepare_phone.sh install_the_magisk_app

To create a new task group with these tasks add the task group defintition below to the file prepare_phone.include. Or, to avoid updating the file prepare_phone.include after each update of that file:

Create a new additional include file with the task group definition:

create install_os_and_magisk.include

cat <<EOT >./install_os_and_magisk.include # avoid warnings from prepare_phone.sh regarding missing DEFAULT_TASKS entries # DEFAULT_TASKS=none # define the new task group with the tasks to install the OS, install Magisk, and enable adb # TASK_GROUP_INSTALL_OS_AND_MAGISK=" enable_disable_abort_on_error:enable boot_phone_from_twrp_image:force # install the new OS # install_os:noreboot:factory_reset:/data/backup/Redmi_9T/lineage-18.1-20210514-UNOFFICIAL-juice.zip # install Magisk only in the boot partition -- do not store any files in /data # install_magisk_in_boot_partition_only:noreboot:next # add the init scripts to enable adb to the ramdisk in the boot partition -- do not store any files in /data # until now nothing was written to the /data partition therefor the reboot into the new OS will succeed # enable_adb:reboot # now the phone booted the first time from the new OS (task parameter "reboot") and the OS initialized the encryption for the /data partition # Therefor we can now write files to /data (either while booted into the OS or while booted into an TWRP image with support for accessing encrypted /data partitions) install_magisk_in_data_adb_only:reboot:copy_apk:use_apk # task necessary because a bug in install_magisk_via_twrp.sh version v3.1.2.0 or older # boot_phone_into_android # install the Magisk app # install_the_magisk_app # optional tasks: enable root access in adb # enable_root_access_for_the_shell:reboot " EOT

Then you can use task group like this

./prepare_phone.sh -i ./install_os_and_magisk.include -i default --list_task_groups INSTALL_OS_AND_MAGISK

or

./prepare_phone.sh -i ?./install_os_and_magisk.include INSTALL_OS_AND_MAGISK

To double check the tasks in the groups do

./prepare_phone.sh -i ./install_os_and_magisk.include -i default --list_task_groups INSTALL_OS_AND_MAGISK

e.g

/data/develop/android/scripts_on_linux ] $ ./prepare_phone.sh -i ./install_os_and_magisk.include -i default --list_task_groups INSTALL_OS_AND_MAGISK

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./prepare_phone.sh -i ./install_os_and_magisk.include -i default --list_task_groups INSTALL_OS_AND_MAGISK [07.10.2023 12:12 ] ### prepare_phone.sh started at 07.10.2023 12:12:10 (The PID of this process is 817680) [07.10.2023 12:12 ] ### Processing the parameter ... [07.10.2023 12:12 ] Not yet used parameter are: "INSTALL_OS_AND_MAGISK" [07.10.2023 12:12 ] Tasks to execute are: "INSTALL_OS_AND_MAGISK" [07.10.2023 12:12 ] Parameter for the function init_tasks are: "" [07.10.2023 12:12 ] ### The logfile used is /var/tmp/prepare_phone.sh.log [07.10.2023 12:12 ] Checking the include file "/data/develop/android/scripts_on_linux/prepare_phone.include" ... [07.10.2023 12:12 ] Reading the include file "/data/develop/android/scripts_on_linux/prepare_phone.include" ... [07.10.2023 12:12 ] Checking the include file "/data/develop/android/scripts_on_linux/install_os_and_magisk.include" ... [07.10.2023 12:12 ] Reading the include file "/data/develop/android/scripts_on_linux/install_os_and_magisk.include" ... Defined Task groups matching one of the regex "INSTALL_OS_AND_MAGISK" are: INSTALL_OS_AND_MAGISK : enable_disable_abort_on_error:enable boot_phone_from_twrp_image:force install_os:noreboot:factory_reset:/data/backup/Redmi_9T/lineage-18.1-20210514-UNOFFICIAL-juice.zip install_magisk_in_boot_partition_only:noreboot:next enable_adb:reboot install_magisk_in_data_adb_only:reboot:copy_apk:use_apk enable_root_access_for_the_shell:reboot install_the_magisk_app [07.10.2023 12:12 ] ### The logfile used was /var/tmp/prepare_phone.sh.log [07.10.2023 12:12 ] ### The start time was 07.10.2023 12:12:10, the script runtime is (day:hour:minute:seconds) 0:00:00:00 (= 0 seconds) [07.10.2023 12:12 ] ### prepare_phone.sh ended at 07.10.2023 12:12:10 (The PID of this process is 817680; the RC is 0) [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Note:

There is a bug in the current version of the script install_magisk_via_twrp.sh :

Either download the current version from here

https://bnsmb.de/files/public/Android/install_magisk_via_twrp.sh

or add the task

boot_phone_into_android

after the task

install_magisk_in_data_adb_only:reboot:copy_apk:use_apk

How to change any file or directory using Magisk

URL: https://xdaforums.com/t/how-to-change-any-file-or-directory-using-magisk.4543103/

How to change any file or directory using Magisk

To change a file (or directory) in one of the read-only mounted filesystems in the Android OS Magisk can be used.
(see How to change files in the directory /system with Magisk and How to make files in /system writable )

But this method only works for files in the directories /system, /vendor, /product, or /system_ext.

In current Android OS implementations there are a lot of other directories on read-only filesystems with files that must be changed or replaced to change the behaviour of Android.

For example on some Android implementations the boot animation is in the file /my_product/media/bootanimation/bootanimation.zip. The standard mechanism to change a file on a read-only mounted filesystem via Magisk can not be used to replace this file with another file.

But Magisk also supports executing scripts while booting the phone (see How to run a script at every boot using Magisk).

And the Magisk init scripts in the directory /data/adb/post-fs-data.d are executed early in the boot process. Therefor we can create a Magisk init script to replace any file with another file using the same method Magisk is using (that is a "mount -o bind ..." ):

Note

All commands in this Howto must be done by the user root.

Example

To replace the boot animation with another one copy the ZIP file with the new boot animation to the directory /data/adb (or any other read-write mounted directory available early in the boot process), e.g:

ASUS_I006D:/data # ls -l /data/adb/Earth_bootanimation.zip-rw-r--r-- 1 u0_a130 media_rw 13227720 2022-04-16 10:58 /data/adb/Earth_bootanimation.zip ASUS_I006D:/data #

Make sure that the owner, group, permissions, and SELinux contexts for the file are okay using these commands:

# get the owner, group, permissions, and SELinux context of the file to replace # ASUS_I006D:/ # ls -lZ /system/media/bootanimation.zip -rw-r--r-- 1 root root u:object_r:system_file:s0 4109852 2009-01-01 01:00 /system/media/bootanimation.zip ASUS_I006D:/ #
# correct the config for the new file # chown root:root /data/adb/Earth_bootanimation.zip chmod 0644 /data/adb/Earth_bootanimation.zip chcon -v u:object_r:system_file:s0 /data/adb/Earth_bootanimation.zip
Result:

ASUS_I006D:/data # ls -ldZ /data/adb/Earth_bootanimation.zip-rw-r--r-- 1 root root u:object_r:system_file:s0 13227720 2022-04-16 10:58 /data/adb/Earth_bootanimation.zip ASUS_I006D:/data #

Now create a Magisk init script to replace the file via bind mount:

echo "mount -o bind /data/adb/Earth_bootanimation.zip /system/media/bootanimation.zip " >post-fs-data.d/change_bootanimation.sh chmod 755 post-fs-data.d/change_bootanimation.sh
That's it .. Reboot the phone and enjoy the new animation.

To remove the new animation just delete the script post-fs-data.d/change_bootanimation.sh and reboot the phone.

The same method can be used to replace a complete directory using these steps:

e.g. to make the directory /odm/etc writable do:

# # create the new directory # mkdir /data/adb/odm_etc # # copy the existing files in /odm/etc to the new directory # cd /odm/etc && find . | cpio -pdum /data/adb/odm_etc # # create the Magisk init script to replace /odm/etc with the new directory /data/adb/odm_etc # echo "# make /odm writable mount -o bind /data/adb/odm_etc /odm/etc " >/data/adb/post-fs-data.d/make_odm_etc_writable chmod 0755 /data/adb/post-fs-data.d/make_odm_etc_writable

Now reboot the phone and check the result:

ASUS_I006D:/ # ASUS_I006D:/ # ls -l /odm/etc total 15 -rw-r--r-- 1 root root 4961 2009-01-01 01:00 NOTICE.xml.gz -rw------- 1 root root 1136 2009-01-01 01:00 build.prop -r--r--r-- 1 root root    0 2009-01-01 01:00 fs_config_dirs -r--r--r-- 1 root root    0 2009-01-01 01:00 fs_config_files -rw-r--r-- 1 root root    0 2009-01-01 01:00 group -rw-r--r-- 1 root root    0 2009-01-01 01:00 passwd drwxr-xr-x 2 root root 3452 2023-01-18 16:30 selinux ASUS_I006D:/ # # # create a new file in the directory used to replace /odm/etc # ASUS_I006D:/ # touch /data/adb/odm_etc/test.$$ASUS_I006D:/ # ls -l /data/adb/odm_etc/test.$$ -rw-r--r-- 1 root root 0 2023-01-18 16:41 /data/adb/odm_etc/test.4597 ASUS_I006D:/ # # # check the contents of the directory /odm/etc # ASUS_I006D:/ # ls -l /odm/etc/test.$$-rw-r--r-- 1 root root 0 2023-01-18 16:41 //odm/etc/test.4597 ASUS_I006D:/ #

To replace files that are used by user apps, make sure that the file that is to replace the original file is readable by any user. I recommend creating the new files in the directory /data/local/tmp - the file system for this directory supports all permissions for Linux files and is readable by any user.

Caution

I'm sure there is a reason why this feature is not implemented for all directories by default in Magisk
And, for example, if you make the directory /odm/etc writable using the method described above Android will complain after the next reboot with an error message that something is wrong with your phone.
Therefor please use with care! If possible only use this approach to make single files writable.

Trouble Shooting

If it looks like the "new" file is not used or something else went wrong, but the phone still boots into the Android system, this could be due to an incorrect SELinux context. To find this kind of error, look for the name of the replaced file in the output of logcat with grep.

To check if an error is caused by an invalid SELinux context you can temporary disable SELinux:

# get the current SELinux mode (0 = disabled = permissive , 1 = enabled) # ASUS_I006D:/ $ getenforceEnforcing ASUS_I006D:/ $ # temporary disable SELinux # ASUS_I006D:/ $ su - -c "setenforce 0"ASUS_I006D:/ $ # check the result # ASUS_I006D:/ $ getenforcePermissive ASUS_I006D:/ $

To enable SELinux again use setenforce with the parameter "1" or reboot the phone. For files that are used early in the boot process the command to disable SELinux temporary should be in a script in /data/adb/post-fs-data.d.

In case the phone does not boot anymore after replacing a file or directory using this method:

    Reboot the phone from a recovery with adb support (like TWRP) and delete the script in /data/adb/post-fs-data to fix the error

Notes

Files or directories that are used by Android before the Magisk init scripts are executed can not be changed using this method. These files must be replaced by changing the files in the ramdisk on the boot partition
(see How to change files in the boot image using Magisk and How to trigger an action when a property is changed )

There are various Magisk Modules available in the internet to replace the animation that use this technique

To test a boot animation while the Android OS is running do:

mount -o bind /data/adb/bootanimation_android12.zip /system/media/bootanimation.zip                                                          bootanimation

Update 11.07.2023 /bs

Added more details about how to configure the SELinux context for the new file

Update 18.03.2024 /bs

Added a hint about the recommended directory to be used for new files

Update 24.06.2023/bs

How to use an overlay filesystem to make filesystems writable

URL: https://xdaforums.com/t/how-to-change-any-file-or-directory-using-magisk.4543103/#post-88679517

How to use an overlay filesystem to make filesystems writable

Another method to change files in read-only filesystems is to use an overlay filesystem.

This method is implemented in the Magisk Module Magisk Overlayfs.
The source code for this Magisk module is available here: https://github.com/HuskyDG/magic_overlayfs

Using this Magisk Module every file in most of the read-only filesystems in Android can be changed; excerpt from the documentation

Make most parts of system partition (/system, /vendor, /product, /system_ext, /odm, /odm_dlkm, /vendor_dlkm, ...) become read-write.

I successfully tested this Magisk Module on an ASUS Zenfone 8 running OmniROM 13.

Notes:

There is no installable zip file in the repository for the Magisk Overlayfs . But the repository contains a script to create the zip file (build.sh)

Note that I had to change the code to create Magisk module zip file in the script build.sh to create an installable Magisk Module:

I replaced the line

zip -r9 out/magisk-module-release.zip out/magisk-module

with

cd out/magisk-module && zip -r9 ../magisk-module-release.zip

How to configure the WiFI in Android via script

URL: https://xdaforums.com/t/how-to-configure-the-wifi-in-android-via-script.4548489/

How to configure the WiFi in Android via script

Update 11.09.2023

see also here How to configure the WiFI in Android via CLI command

In the current version of Android the WiFi configuration is stored in the file

/data/misc/apexdata/com.android.wifi/WifiConfigStore.xml

e.g.

ASUS_I006D:/ # ls -lZtr /data/misc/apexdata/com.android.wifi/WifiConfigStore.xml-rw------- 1 system system u:object_r:apex_system_server_data_file:s0 4884 2023-01-29 08:55 /data/misc/apexdata/com.android.wifi/WifiConfigStore.xml ASUS_I006D:/ #

The file is in plain ASCII xml format so it can be processed by any executable to change text files (editor, sed, etc).
But IMHO it's better to configure the WiFi manually via GUI and then use the file WifiConfigStore.xml with that configuration for configuring the WiFi via script.

Therefor I use this code in the post install script to install and configure my phone (see How to install and configure the Android OS via Script):

script commands to enable and configure WiFi

# # sample post install script for the customizing of the phone # # This script will be copied to the phone and executed there # # The script is executed by the user shell; use "su - -c <command>" to execute commands as user root # echo "" echo "*** Postinstall script is running ..." echo "" # ... other customizations .... # # create the WiFi config file # # -rw------- 1 system system u:object_r:apex_system_server_data_file:s0 4884 2023-01-29 08:55 /data/misc/apexdata/com.android.wifi/WifiConfigStore.xml # WIFI_CONFIG_FILE="/data/misc/apexdata/com.android.wifi/WifiConfigStore.xml" WIFI_CONFIG_FILE_BACKUP="${WIFI_CONFIG_FILE}.$$.bkp" NEW_WIFI_CONFIG_FILE="/sdcard/Download/WifiConfigStore.xml" if [ -r "${WIFI_CONFIG_FILE}" ] ; then echo "Creating a backup of the file \"${WIFI_CONFIG_FILE}\" in \"${WIFI_CONFIG_FILE_BACKUP}\" ...." cp "${WIFI_CONFIG_FILE}" "${WIFI_CONFIG_FILE_BACKUP}" fi echo "Creating the file \"${NEW_WIFI_CONFIG_FILE}\" ..." cat >"${NEW_WIFI_CONFIG_FILE}" <<-\EOT *** add here the contents of the file /data/misc/apexdata/com.android.wifi/WifiConfigStore.xml from the phone after manually configuring the WLAN EOT if [ $? -eq 0 ] ; then echo "Creating the file \"${WIFI_CONFIG_FILE}\" ..." su - -c cp "${NEW_WIFI_CONFIG_FILE}" "${WIFI_CONFIG_FILE}" && \ su - -c chmod 600 "${WIFI_CONFIG_FILE}" && \ su - -c chown system:system "${WIFI_CONFIG_FILE}" && \ su - -c chcon -v "u:object_r:apex_system_server_data_file:s0" "${WIFI_CONFIG_FILE}" if [ $? -ne 0 ] ; then echo "Error creating the file \"${WIFI_CONFIG_FILE}\" " fi else echo "Error creating the file \"${NEW_WIFI_CONFIG_FILE}\" " fi # enable WiFi # echo "Enabling WiFi ..." svc wifi enable # optional: Disable mobile data connections # echo "Disabling mobile data ..." svc data disable # Now reboot the phone to activate the new WiFi config .. # echo "Waiting 15 seconds now before rebooting - press CTRL-C to abort ...." i=0 while [ $i -lt 15 ] ; do (( i = i + 1 )) printf "." sleep 1 done printf "\n" echo "Now rebooting the phone ..." reboot #

Notes:

Be aware the the file /data/misc/apexdata/com.android.wifi/WifiConfigStore.xml contains the passwords for all configured WLANs in plain text.

Update 29.06.2024

Note that the factory default MAC address is also defined in that file:

ASUS_I006D:/storage/emulated/0/Download/install # grep mac_address /data/misc/apexdata/com.android.wifi/WifiConfigStore.xml <string name="wifi_sta_factory_mac_address">7c:14:c7:77:8b:c3</string> ASUS_I006D:/storage/emulated/0/Download/install #

There should be a command to force Android to reread the file WifIConfigStore.xml but I don't know that.

Android can be forced to re-read the file WifIConfigStore.xml by killing the process system_server, e.g:

pkill system_server

But that is more or less a warm reboot and I do not know what other side effects that restart triggers.

There is an app to configure WiFi via adb command:

https://github.com/steinwurf/adb-join-wifi

example:

adb shell am start -n com.steinwurf.adbjoinwifi/.MainActivity -e ssid myssid -e password mywlan_password -e password_type WPA

The app works but unfortunately it does not support enabling the setting to use the device MAC instead of the random MAC so it's not usable in my environment.

Another app to configure WiFi via adb command is available here:

https://github.com/pr4bh4sh/adb-wifi-setting-manager

Unfortunately that app also does not support enabling choosing the MAC address for the connection.

There is also a script to convert an old Android wpa_supplicant.conf file to the newer (post-Oreo) WifiConfigStore.xml file (not tested):

https://github.com/mnalis/android-wifi-upgrade

It's also possible to use the command wpa_cli to configure WiFi but be aware the wpa_cli only works if selinux is (temporary) disabled, example:

wpa_cli example

ASUS_I006D:/ $ su - ASUS_I006D:/ # ASUS_I006D:/ # setenforce 0 ASUS_I006D:/ # ASUS_I006D:/ # ASUS_I006D:/ # wpa_cli wpa_cli v2.10-devel-11 Copyright (c) 2004-2019, Jouni Malinen <j@w1.fi> and contributors This software may be distributed under the terms of the BSD license. See README for more details. Using interface 'wlan0' Interactive mode <3>Control interface command 'BSS RANGE=ALL MASK=0x2' <3>Control interface command 'LIST_CREDS' <3>Control interface command 'LIST_NETWORKS' <3>Control interface command 'STA-FIRST' > > list_networks network id / ssid / bssid / flags 0 Zeilsheim any [CURRENT] 1 any [DISABLED] <3>Control interface command 'LIST_NETWORKS' > > quit ASUS_I006D:/ #

Note: Use help to get list of known commands

Test Environment

I tested the WiFi config via script described above in these Android distributions:

OmniROM 13 (Android 13)
ASUS Android 12
ASUS Android 13

This method seems not to work in Android 11

How to configure the WiFI in Android via CLI command

URL: https://xdaforums.com/t/how-to-configure-the-wifi-in-android-via-cli-command.4626435/

How to configure the WiFI in Android via CLI command

In the forum entry How to configure the WiFi in Android via Script I described some methods to configure the WLAN in Android via script. In the end I recommended to configure the WLAN manually via the GUI and then save the configuration file to copy it back later for a new configuration.

This was mainly because I couldn't find a way to define the MAC address to be used for configuring the WLAN interface.

Now I know that this can be done using another Android cli command:
cmd wifi connect-network

e.g.:

cmd wifi connect-network mySSID wpa2 mypassword

To configure the randomization scheme for the MAC address (use the fixed MAC address or a random MAC address) the parameter -r of the wifi cmd wifi sub command connect-network can be used:

-r auto|none|persistent|non_persistent - MAC randomization scheme for the network

Using that command it's possible to configure the WLAN interface via script and the usage of the GUI is not necessary anymore.

Notes:

The command cmd wifi must be executed by the user root.

Use
cmd wifi

to get the usage help for the command

The syntax for the cmd wifi sub command connect-network is:

connect-network <ssid> open|owe|wpa2|wpa3 [<passphrase>] [-x] [-m] [-d] [-b <bssid>] [-r auto|none|persistent|non_persistent] Connect to a network with provided params and add to saved networks list <ssid> - SSID of the network open|owe|wpa2|wpa3 - Security type of the network. - Use 'open' or 'owe' for networks with no passphrase - 'open' - Open networks (Most prevalent) - 'owe' - Enhanced open networks - Use 'wpa2' or 'wpa3' for networks with passphrase - 'wpa2' - WPA-2 PSK networks (Most prevalent) - 'wpa3' - WPA-3 PSK networks -x - Specifies the SSID as hex digits instead of plain text -m - Mark the network metered. -d - Mark the network autojoin disabled. -h - Mark the network hidden. -p - Mark the network private (not shared). -b <bssid> - Set specific BSSID. -r auto|none|persistent|non_persistent - MAC randomization scheme for the network

Test Environment

Tested with this Android versions:

OmniROM 13
ASUS Android 13

see also here https://stackoverflow.com/questions/75294766/android-device-how-to-connect-wifi-by-adb-shell-cmd-wifi-command

How to install Magisk v26.0 or newer via script

URL: https://xdaforums.com/t/how-to-install-magisk-into-the-boot-partition-using-a-script.4456621/

How to install Magisk v26.0 or newer via script

In Magisk v26.0 the developer changed the code so that it's not possible anymore to install Magisk into the boot partition while booted from a recovery image.

from the Magisk change log:

"This means that any installation of Magisk v26+ using custom recovery will be incomplete; a subsequent re-installation through the Magisk app after booting up is required."

The installation of Magisk v26.x while booted into the recovery (for example using my script install_magisk_via_twrp.sh; see How to install Magisk into the boot partition using script) will succeed but executing the Magisk app after the necessary reboot will end with this dialog:

After pressing OK in that dialog Magisk will reinstall itself in the boot partition and reboot. And afterwards Magisk works again. But that step needs user intervention.

Without executing this additional step Magisk won't work. There is no way to trigger this reinstallation via an CLI command that is usable in a script (at least I did not found it until now ...)

Unfortunately, Magisk also does not execute the init scripts from /data/adb/post-fs-data.d or /data/adb/service.d in this state.

Therefor the automatic installation of Android, TWRP, Magisk, etc like implemented in the tasks in prepare_phone.include (see How to install and configure the Android OS for details) will fail.

To get around this I added some code to the script install_magisk_via_twrp.sh to support the installation of Magisk v26:

install_magisk_via_twrp.sh version v3.1.0.0 (or newer) can now be used to also install Magisk v26 into the boot partition. There are two new parameter for the script install_magisk_via_twrp.sh to select the method to use for the installation:

oldmethod - install Magisk into the boot partition while booted into a recovery

newmethod - install Magisk into the boot partition image while booted into the Android OS

The installation using the "newmethod" only works if their is a working adb connection to the running Android OS on the phone.

Without one of the two new parameter the script will check the Magisk apk file to detect the necessary method to install Magisk into the boot partition. The default method if no apk file is used is still the old method.

Patching the boot image with Magisk v26 is done using these steps:

boot the phone from a TWRP image
create an image of the boot partition while booted from the TWRP image
copy the boot partition image and the Magisk executables to a directory that can be accessed by the user shell while booted into the Anroid OS
boot the phone into the Android OS
patch the boot partition image using the Magisk binaries (that can be done by the non-root user shell)
reboot the phone from a TWRP image
install the patched boot image via dd or fastboot

To do an automatic installation via script using Magisk v26 I added another task in the file prepare_phone.include:

install_magisk_v26_in_boot_partition

This task will install Magisk using the new method described above.

Because the installation of Magisk v26 needs a working adb connection the tasks in prepare_phone.include will first install Magisk v25.x and enable adb via Magisk init script like before. After this is done Magisk v26 can be installed via script.

So the new task list to install and configure the Android OS in the file prepare_phone.include is:

DEFAULT_TASKS=" enable_disable_abort_on_error:enable boot_phone_from_twrp_image install_os:noreboot:factory_reset install_twrp:noreboot:next # # first install Magisk v25.x into the boot partition; this installation is necessary to # enable the access via adb to the phone; the Magisk installation will be overwritten later # with Magisk v26 or newer # install_magisk_in_boot_partition_only:noreboot:nextenable_adb:reboot # install_magisk_in_data_adb_only:noreboot:copy_apk enable_root_access_for_the_shell:reboot wait_for_access_via_adb:120:kill # # the next step is necessary to install Magisk v26 or newer # install_magisk_v26_in_boot_partition:active:reboot# install_the_magisk_app wait_for_access_via_adb:120:kill enable_disable_abort_on_error:disable install_essential_scripts install_essential_apps install_essential_magisk_modules enable_root_access_for_apps:reboot execute_script_on_the_phone:ignorerc "

To distinguish between the apk files for Magisk v25 and Magisk v26 two new variables must be defined in the config file prepare_phone.conf for the script prepare_phone.include:

MAGISK_v26_APK_FILE="/data/backup/Android/EssentialApps/Magisk-v26.1.apk" MAGISK_v25_APK_FILE="/data/backup/Android/EssentialApps_Backup/Magisk-v25.2.apk"

see here for the log of an update of Magisk v25 to Magisk v26 via script using the command

./prepare_phone.sh install_magisk_v26_in_boot_partition

see here for the log of an automated installation using the tasks in prepare_phone.include with the installation of Magisk v26 using the command

./prepare_phone.sh all

Testenvironments for the installation of Magisk v26 via script

an ASUS Zenfone 8 running

OmniROM 13 with MicroG and Magisk v26.1
OmniROM 12 with MicroG and Magisk v26.1
ASUS Android 13 and Magisk v26.1
ASUS Android 12 and Magisk v26.1
LineageOS 19 and Magisk v26.1

All tests were done using TWRP 3.7.0.12

How to recover data from a phone a with non-working touch screen

URL: https://xdaforums.com/t/how-to-recover-data-from-a-phone-with-a-non-working-touch-screen.4578795/

How to recover data from a phone with a non-working touch screen

If the touch screen of a phone is not working anymore you might be able to fetch the data from the phone using this approach:

1. boot the phone into the bootloader using the power button and the volume up and volume down button

2. reboot the phone from a TWRP image for the phone using fastboot:

fastboot boot <twrp_image_file>

3. connect to the phone via adb and use the command twrp to decrypt the encrypted partition:

twrp decrypt password

(see the TWRP documentation for the syntax for the password)

Example:

# # -> without entering the password the data partition is encrypted: # ASUS_I006D:/ # ls /data/media/0/ +lnodCAAAAAnYFLObBoWS5ZNb5yVMpze DWk4KDAAAAA,B1xVDmT3dZI87g2C0IS8 M6jcmCAAAAAU,4gn9o0zoBZmkysyXy49 haPcnBAAAAw9o2lLF97zpPTaC+byP92s 0nUvhAAAAAQ9ptz+L5qPQrpzcAt4G06K EcHrdAAAAAglBtbhNWl0NvyX8aW+3CHN O5ZVrDAAAAAlKd7PPpo7CaQmkFbZzKRm mSKc8BAAAAwm+12sMClHPf+zwtHAeW+Q 4nFjMBAAAAwv2oUczXEpKUDfRIOi+nR5 FA1aZAAAAAg2XNF+M859lpPY,3rUC2cY Vda+LCAAAAwR+g6Tav35+h,Epc8latGR rMJEjAAAAAgwTgx,vA5ewWMVizc2LAos 5yL8gCAAAAgekBDBUi4,W78rJAkgFcMF FHf6sBAAAAAvsMrihjbmRxtr4wiLWjgw VvEV4DAAAAgRgoqCfrypNqryZxRACnfr xfaFHCAAAAwvXPP3IbMXPgyWUPMSXi+z 8CsF+DAAAAwI3tImfOqIS9aHCQ3S7Cdg JpuFgAAAAAwdEzuQerx4681dzGLjlK3X W9W9SCAAAAgh6lwoP0mNiEK9VIWJoH6L 8KdHMCAAAAQOgtDHq,XdSfFPW+XURA2M KlfTJCAAAAwL93jkH2ugnU4eUG3Zzm7M bE7t0CAAAAQ2YuHL9styNaf,iSf2cCO, ASUS_I006D:/ # # # decrypt the data partition using TWRP (12345 is the password in this example) # ASUS_I006D:/ # twrp decrypt 12345 Attempting to decrypt data partition or user data via command line. Attempting to decrypt FBE for user 0... User 0 Decrypted Successfully Data successfully decrypted Updating partition details... ...done ASUS_I006D:/ # # # Now the files in the data partition can be read and copied to the PC using for example adb pull # ASUS_I006D:/ # ls /data/media/0/Alarms Audiobooks Documents Movies Notifications Podcasts Ringtones test.img Android DCIM Download Music Pictures Recordings TWRP test1.img ASUS_I006D:/ #

If you're running Linux on the PC you could use tar to download all files from the phone, e.g.

OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android ] $ adb shell tar -czf - /data/media/0 | cat >./data_on_the_phone.tar removing leading '/' from member names [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android ] $ [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android ] $ tar -tvf ./data_on_the_phone.tar | head -10 drwxrws--- media_rw/media_rw 0 2023-04-23 17:27 data/media/0 drwxrws--x media_rw/media_rw 0 2023-04-22 12:31 data/media/0/Android/ drwxrws--x media_rw/ext_data_rw 0 2023-04-22 13:15 data/media/0/Android/data/ -rw-rw---- u0_a116/ext_data_rw 0 2023-04-22 12:31 data/media/0/Android/data/.nomedia drwxrws--- u0_a143/ext_data_rw 0 2023-04-22 13:12 data/media/0/Android/data/org.fdroid.fdroid/ drwxrws--- u0_a143/ext_data_rw 0 2023-04-22 13:12 data/media/0/Android/data/org.fdroid.fdroid/files/ drwxrws--- u0_a146/ext_data_rw 0 2023-04-22 13:12 data/media/0/Android/data/com.mixplorer/ drwxrws--- u0_a146/ext_data_rw 0 2023-04-22 13:15 data/media/0/Android/data/com.mixplorer/cache/ drwxrws--- u0_a146/ext_data_rw 0 2023-04-22 13:15 data/media/0/Android/data/com.mixplorer/cache/temp/ drwxrws--- u0_a146/ext_data_rw 0 2023-04-22 13:17 data/media/0/Android/data/com.mixplorer/cache/log/ [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android ] $

How to re-install the Android OS from ASUS on the ASUS Zenfone 8

URL: https://xdaforums.com/t/asus-zenfone-8-how-to-rollback-from-lineageos-to-original-rom.4583833/#post-88525935

How to re-install the Android OS from ASUS on the ASUS Zenfone 8

Update 12.01.2024

As of 17.08.2023 the firmware for the ASUS Zenfone 8 can not be downloaded from the ASUS web Page anymore.

see ASUS Zenfone 8 Firmware Download for alternatives for downloading the firmware.

How can I rollback to original Asus ROM without a need of a computer, can I do that with the Recovery mode using the Bootloader (Power button + Volume UP) then select reset to factory setting ?

I doubt that you can rollback without using a PC (how did you install the LineageOS without a Computer?)

The Android OS images from ASUS for the Zenfone 8 can be downloaded from here:

https://www.asus.com/de/Mobile/Phones/ZenFone/Zenfone-8/HelpDesk_BIOS/

In principal it should be possible to install them using fastboot commands or a recovery from one of the CustomROMs or TWRP (https://twrp.me/) but in my tests most of the time that did not work.

Therefor I recommend using a raw image to reinstall the Android OS from ASUS like described in this thread:

https://xdaforums.com/t/full-recover-to-stock-if-things-went-really-bad.4337467/

Note that installing a raw image will wipe all data on the phone!

The raw image for Android 13 for the ASUS Zenfone 8 is the image for the beta version of Android 13 on this page:

https://www.asus.com/Content/Android-13-Beta/

After the installation of the Beta image you should install the official Android 13 using the current Android 13 OS image for the Zenfone 8 from this page:

https://www.asus.com/de/Mobile/Phones/ZenFone/Zenfone-8/HelpDesk_BIOS/

The raw image for Android 12 for the ASUS Zenfone 8 can be found here (at the bottom of the page):

https://www.asus.com/Content/Android-13-Beta/

The raw image for Android 11 for the ASUS Zenfone 8 can be be found here (at the bottom of the page):

https://www.asus.com/Content/Android-12-Beta/

After the installation of one of the raw images the installation of the current version of the Android OS from ASUS via the "normal" method is strongly recommended.

How to properly install the GApps without using a computer ? can I download it as APK and install it ?

GApps are partially system apps and must still be installed while the Android OS is not running (e.g. while booted from a recovery like TWRP)

If you do not really need all the Google apps but only the Google stack to get an appication running you could test if MicroG (a replacement for the Google Stack; see https://microg.org/) is sufficient for your application. MicroG seems to be a bit outdated but it isn't -it's still working fine.
I use it on my phone to be able to use Apps that require Google functionality but without installing the Google Stack (and some of the Google Apps I like for example. Google Maps)

To use MicroG either install a CustomROM with MicroG already included like the LinegaOS with MicroG (https://download.lineage.microg.org/sake/) or (my prefered ROM) the OmniROM with MicroG (https://dl.omnirom.org/zenfone8/).

Or install the MicroG packages on your LineageOS either manually (not recommended) or using an installation image for MicroG like for example NanoDroid (https://nanolx.org/nanolx/nanodroid/).
One advantage of NanoDroid is that it also contains a patched Playstore that does work with MicroG.

How to Install TWRP into the boot partition again after the installation of an OS update

URL: https://xdaforums.com/t/installing-twrp-into-the-boot-partition-again-after-the-installation-of-an-os-update.4586725/

How to Install TWRP into the boot partition again after the installation of an OS update

After installing an operating system update on a phone with the Android operating system and A/B slots, Magisk can be installed in the boot partition of the inactive slot with the new operating system version using the "Install to inactive slot" installation option in the Magisk App.

Unfortunately this option is not available for TWRP.

Therefor I wrote a litlle script to install TWRP into the boot partition of the inactive slot with the new operating system version after an OS upgrade.
The method works on my ASUS Zenfone 8 and should in principle also work on other phones with the same architecture (Qualcomm CPU, A/B slots, and boot partitions with ram disks).

Looking at the source code for TWRP (https://github.com/TeamWin/android_bootable_recovery/blob/android-12.1/twrpRepacker.cpp) it can be seen that the installation of TWRP is in principle only the installation of a new ramdisk:

TWRP source code excerpt

bool twrpRepacker::Flash_Current_Twrp() { if (!TWFunc::Path_Exists("/ramdisk-files.txt")) { LOGERR("can not find ramdisk-files.txt"); return false; } PartitionManager.Unlock_Block_Partitions(); Repack_Options_struct Repack_Options; Repack_Options.Disable_Verity = false; Repack_Options.Disable_Force_Encrypt = false; Repack_Options.Type = REPLACE_RAMDISK_UNPACKED; Repack_Options.Backup_First = DataManager::GetIntValue("tw_repack_backup_first") != 0; std::string verifyfiles = "cd / && sha256sum --status -c ramdisk-files.sha256sum"; if (TWFunc::Exec_Cmd(verifyfiles) != 0) { gui_msg(Msg(msg::kError, "modified_ramdisk_error=ramdisk files have been modified, unable to create ramdisk to flash, fastboot boot twrp and try this option again or use the Install Recovery Ramdisk option.")); return false; } std::string command = "cd / && /system/bin/cpio -H newc -o < ramdisk-files.txt > /tmp/currentramdisk.cpio && /system/bin/gzip -f /tmp/currentramdisk.cpio"; if (TWFunc::Exec_Cmd(command) != 0) { gui_msg(Msg(msg::kError, "create_ramdisk_error=failed to create ramdisk to flash.")); return false; } if (!Repack_Image_And_Flash("/tmp/currentramdisk.cpio.gz", Repack_Options)) return false; else return true; }

Therefor to install TWRP via script after an OS version update two steps are neccessary:

Step one

First we must create the ramdisk with TWRP; this step must only be done once for each TWRP version.

Boot the phone from the TWRP image (NOT from a TWRP installed in the recovery!), e.g:

adb rebboot bootloader sudo fastboot boot /data/backup/ASUS_ZENFONE8/twrp/twrp-3.7.0_12-0-I006D-enhanced.img
Then open an adb session to the running TWRP on the phone and execute:

# create the RAM disk # cd / && sha256sum --status -c ramdisk-files.sha256sum if [ $? -eq 0 ] ; then cd / && /system/bin/cpio -H newc -o < ramdisk-files.txt > /tmp/ramdisk_with_twrp.cpio && /system/bin/gzip -f /tmp/ramdisk_with_twrp.cpio fi # copy the RAM disk and magiskboot to a directory on the internal storage that supports the execution permission bit (not /sdcard/<something>) # mkdir -p /data/develop/twrp cp /tmp/ramdisk_with_twrp.cpio.gz /data/develop/twrp/ramdisk_with_twrp.cpio cp /bin/magiskboot /data/develop/twrp/

Notes:

It is recommended to store the files ramdisk_with_twrp.cpio and magiskboot also on the PC.
The executable magiskboot is only required if Magisk is not installed on the phone.

Now copy the script install_twrp_ramdisk.sh to the phone; this script can be copied to any filesystem with write access on the phone.

That's it .

Step two

The next time an OS update should be done using these steps:

1. install the new OS version either manually or via OTA but do NOT reboot afterwards.

2. execute the script install_twrp_ramdisk.sh to install TWRP into the boot partition for the inactive slot either in an shell on the phone or in an adb shell, e.g. (sh is only necessary if the script is on a filesystem that does not support the execution permission)
sh /sdcard/Download/install_twrp_ramdisk.sh

Note that the script needs root access to read and write the boot partition

3. optional: Install Magisk into the new boot partition using the install option "Install to inactive slot" in the Magisk App

4. reboot the phone to activate the new OS version

Test Environment

ASUS Zenfone 8
OmniROM 13
Magisk v25.2
TWRP 3.7.0

Notes

Set the environment variable WORK_DIR before executing the script to use another directory with the image
Set the environment variable TWRP_RAMDISK_NAME before executing the script to use another filename for the ram disk image.

The usage for the script install_twrp_ramdisk.sh is:

install_twrp_ramdisk.sh -H

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ksh ../scripts/install_twrp_ramdisk.sh -H Usage: install_twrp_ramdisk.sh [-h|-H] This script must run in a shell on the phone (or in an adb shell) after an OS update was installed (either manual or via OTA) but before rebooting the phone. The script will then install TWRP into the inactive slot. The script must be executed by the user "root"; if it's executed by a non-root user it will restart itself using "su - -c scriptname" Supported environment variables WORK_DIR directory with the ram disk image and probably the executable magiskboot TWRP_RAMDISK_NAME name of the file with the ram disk image (without path) Prerequisites The script requires these files: /data/develop/twrp/ramdisk_with_twrp.cpio - the ram disk with TWRP /data/develop/twrp/magiskboot or /data/adb/magisk/magiskboot - the magiskboot executable To create the ram disk with TWRP do: Boot the phone from the TWRP image (NOT from a TWRP installed in the recovery!), open an adb session to the running TWRP on the phone and execute: cd / && sha256sum --status -c ramdisk-files.sha256sum if [ $? -eq 0 ] ; then cd / && /system/bin/cpio -H newc -o < ramdisk-files.txt > /tmp/ramdisk_with_twrp.cpio && /system/bin/gzip -f /tmp/ramdisk_with_twrp.cpio fi mkdir -p /data/develop/twrp cp /tmp/ramdisk_with_twrp.cpio.gz /data/develop/twrp/ramdisk_with_twrp.cpio cp /bin/magiskboot /data/develop/twrp/ Note: It is recommended to store the files ramdisk_with_twrp.cpio and magiskboot also on the PC. The file magiskboot is only required if Magisk is not installed on the phone. The directory used for the files can be any directory on the phone on a filesystem supporting the execute permission bit. But do not forget to change the directory used in the script below (variable WORK_DIR) also. [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

see How to add additional files to an TWRP image for how to add additional files to the TWRP installation

see How to install TWRP via script for how to install TWRP with a script running on the PC
see https://github.com/Magisk-Modules-Repo/twrp-keep for a Magisk module with a script to reinstall TWRP into the inactive boot partition after an OS Udate

Update 11.06.2023

I successfully tested the Magisk Module twrp-keep on an ASUS Zenfone 8 with OmniROM 13, TRWP 3.7.0_12, and Magisk v26.2.

ToDos:

Create an App with GUI to execute the script

Hints for adding files to the boot partition

URL: https://xdaforums.com/t/how-to-change-files-in-the-boot-image-using-magisk.4495645/#post-88571069

Hints for adding files to the boot partition

Question from a forum entry in the XDA forum:

I want to add custom.init.rc with this code

```
./magiskboot unpack boot_root.img
./magiskboot cpio ramdisk.cpio \
"mkdir 0700 overlay.d" \
"add 0700 overlay.d/init.custom.rc init.custom.rc" \
"mkdir 0700 overlay.d/sbin" \
"add 0700 overlay.d/sbin/custom.sh init.custom.sh"
./magiskboot repack boot_root.img
```

After reboot my phone doing nothing, check file /sbin/custom.sh is existed, but /init.custom.rc not exist.
A10 Pixel Experience Redmi4x

That work's like designed. Excerpt from the Magisk developer guide https://topjohnwu.github.io/Magisk/guides.html:

Each *.rc file (except for init.rc) in overlay.d will be read and concatenated AFTER init.rc if it does not exist in the root directory, otherwise it will REPLACE the existing one.

The *rc files will be read and processed while booting the Android OS -- they will not be copied to the boot partition. And that make sense because the *rc files in Android are not use at all after the boot process finished.

To be able to test if an additional *rc file is processed in the running Android OS I recommend to add instructions to set a new property while booting the OS, e.g.:

init.custom.rc

ASUS_I006D:/data/develop # cat init.custom.rc on property:bs.test.prop01=1 setprop bs.test.prop02 "1" on property:bs.test.prop01=0 setprop bs.test.prop02 "0" service test_service_001 /system/bin/sh /data/develop/init001.custom.sh user root group root disabled seclabel u:r:magisk:s0 oneshot service test_service_002 /system/bin/sh ${MAGISKTMP}/init002.custom.sh user root group root disabled seclabel u:r:magisk:s0 oneshot on property:bs.test.prop01=9 exec u:r:magisk:s0 -- /system/bin/sh /data/develop/init.custom001.sh 001 on property:bs.test.prop01=8 exec u:r:magisk:s0 -- /system/bin/sh ${MAGISKTMP}/init.custom002.sh 000 on property:bs.test.prop02=10 start test_service_001 on property:bs.test.prop02=20 start test_service_002 on early-init setprop bs.test.prop01 "111" setprop bs.test.prop02 "222" ASUS_I006D:/data/develop #

The commands in the section "on early-init" in the *rc file can be used to check if the *rc file was processed:

ASUS_I006D:/ $ uptime 21:14:00 up 0 min, 0 users, load average: 1.86, 0.44, 0.14 ASUS_I006D:/ $ getprop bs.test.prop01 111 ASUS_I006D:/ $ getprop bs.test.prop02222 ASUS_I006D:/ $
If the properties are not set the *rc file was not processed. Most probably there is a syntax error in that file

And for the records:

Note that the new properties can only be read by non-root user; non-root users can not change them:

ASUS_I006D:/ $ id uid=2000(shell) gid=2000(shell) groups=2000(shell),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),1078(ext_data_rw),1079(ext_obb_rw),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats),3009(readproc),3011(uhid),3012(readtracefs) context=u:r:shell:s0 ASUS_I006D:/ $ setprop bs.test.prop02 999Failed to set property 'bs.test.prop02' to '999'. See dmesg for error reason. 1|ASUS_I006D:/ $

The new property can only be changed by the user root:

ASUS_I006D:/ $ su - -c setprop bs.test.prop02 999ASUS_I006D:/ $ ASUS_I006D:/ $ getprop bs.test.prop02 999 ASUS_I006D:/ $

To enable non-root user to change these new properties a correct SELinux context must be defined for the properties.

You should also be aware of this issue - again an excerpt from the Magisk Developer Guide https://topjohnwu.github.io/Magisk/guides.html:

Starting from Android 11, the /sbin folder may no longer exists, and in that scenario, Magisk randomly generates a different tmpfs folder each boot. Every occurrence of the pattern ${MAGISKTMP} in your *.rc scripts will be replaced with the Magisk tmpfs folder when magiskinit injects it into init.rc. On pre Android 11 devices, ${MAGISKTMP} will simply be replaced with /sbin, so NEVER hardcode /sbin in the *.rc scripts when referencing these additional files.

To get the value of ${MAGISKTMP} in the running Android OS use the command magisk --path, e.g.

1|ASUS_I006D:/ $ su - -c magisk --path/dev/zunyqjgsazhw ASUS_I006D:/ $

How to install a more recent version of the Playstore in the OmniROM

URL: https://xdaforums.com/t/how-to-install-a-more-recent-version-of-the-playstore-in-the-omnirom.4593203/

How to install a more recent version of the Playstore in the OmniROM

In this forum entry

How to replace the Fake Store from OmniROM with MicroG with a patched Playstore

I described how to replace the fake store in the OmniROM with MicroG with a patched Playstore.

The Playstore version contained in the Magisk module PlayStore_for_MicroG.zip mentioned in that forum entry is quite old: 20.9.20-all. It works (at least for my purpose...) but for others it might be to old.

To install a newer version of the Playstore the Magisk module microG-GApps can be used: The version of the Playstore in this Magisk module is 35.2.19-21.

The Magisk module microG-GApps also contains the necessary MicroG apks for using the Playstore so it can be installed in the OmniROM image without MicroG. in this case be sure to configure the MicroG settings manually before starting the Playstore.
Note that the customizing script in this Magisk module downloads the MicroG packages from the MicroG website so a working internet connection is necessary to install the module

It's also possible to install the Magisk module microG-GApps in the OmniROM with MicroG:

To use the Magisk module microg-GApps in the OmniROM with MicroG it's necessary to disable the fake store from the OmniROM:

This can be done with the Magisk module disable_fake_store.zip.

After installing that module install the Magisk module microG-GApps and reboot the phone to activate both Magisk modules.

The Magisk module microG-GApps will also install additional Google packages and a new version of the MicroG packages (as of 04.06.2023 this is the version 0.2.28.231657) . To remove these optional files either remove them from the ZIP with the Magisk module microG-GApps (and correct the script customize.sh in te ZIP file) or delete the files installed on the phone for these components after installing the module:

The optional files that can be deleted on the phone running the OmniROM with MicroG are:

To remove the additional Google packages delete these files in /data/adb/modules/microG-GApps/system:

./priv-app/GoogleCalendarSyncAdapter
./priv-app/GoogleCalendarSyncAdapter/base.apk
./priv-app/GoogleContactsSyncAdapter
./priv-app/GoogleContactsSyncAdapter/base.apk
./etc/permissions/com.google.android.syncadapters.contacts.xml
./etc/permissions/com.google.android.syncadapters.calendar.xml

To remove the MicroG packages delete these files in /data/adb/modules/microG-GApps/system:

./priv-app/microGServicesCore
./priv-app/microGServicesCore/base.apk
./priv-app/microGServicesFrameworkProxy
./priv-app/microGServicesFrameworkProxy/base.apk
./framework/com.google.android.maps.jar
./etc/permissions/com.google.android.syncadapters.contacts.xml
./etc/permissions/com.google.android.syncadapters.calendar.xml

Before using the Playstore check the output in the Self Check in the MicroG app to make sure everything is still okay. In addition, the "Google Device registration" must be enabled in the MicroG settings.

Trouble Shooting

In case you get the error DF-DFERH-01 after the Google sign in double check the settings in MicroG. If everyhing is ok in the MicroG settings stop the Playstore using "Force stop" in the System application and delete the storage for the Playstore using "Clear storage" in the System application. When done reopen the Playstore.

Update 08.07.2023

If the Playstore does not work after upgrading from the old Magisk Module (e.g. there is no Playstore icon in the menu) execute these commands as user root to fix it:

# 08.07.2023 /bs : Deleted the "echo" command before "pm install"
#
PHONESKY_APK="/system/priv-app/Phonesky/Phonesky.apk" PKG_SIZE=$( ls -l ${PHONESKY_APK} | awk ' { print $5 }' ) cat "${PHONESKY_APK}" | pm install -S ${PKG_SIZE}

How to upgrade the OS with another Android "distribution"

URL: https://xdaforums.com/t/how-to-upgrade-the-os-with-another-android-distribution.4595049/

How to upgrade the OS with another Android distribution

While doing my tests and development with the OmniROM OS it's sometimes useful to update an installed original OmniROM to one of my self-compiled OmniROM images without reinstalling the phone from scratch.

Doing this using the local file feature of the SystemUpdate app from the OmniROM will fail with the error message "Updated failed with error 26". This is a not really understandable error message but checking the output of logcat will print some more infos about the error:

update_engine_error messages

ASUS_I006D:/ $ logcat -d | grep update_engine 06-11 17:08:46.906 1356 1356 I update_engine: [INFO:update_attempter_android.cc(567)] Enabling performance mode. 06-11 17:08:46.913 1356 1356 I update_engine: [INFO:update_attempter_android.cc(322)] Using this install plan: 06-11 17:08:46.914 1356 1356 I update_engine: [INFO:install_plan.cc(80)] InstallPlan: 06-11 17:08:46.914 1356 1356 I update_engine: type: new_update 06-11 17:08:46.914 1356 1356 I update_engine: version: 06-11 17:08:46.914 1356 1356 I update_engine: source_slot: A 06-11 17:08:46.914 1356 1356 I update_engine: target_slot: B 06-11 17:08:46.914 1356 1356 I update_engine: initial url: file:///sdcard/Download/omni-13-20230611-zenfone8-MICROG.zip 06-11 17:08:46.914 1356 1356 I update_engine: hash_checks_mandatory: true 06-11 17:08:46.914 1356 1356 I update_engine: powerwash_required: false 06-11 17:08:46.914 1356 1356 I update_engine: switch_slot_on_reboot: true 06-11 17:08:46.914 1356 1356 I update_engine: run_post_install: true 06-11 17:08:46.914 1356 1356 I update_engine: is_rollback: false 06-11 17:08:46.914 1356 1356 I update_engine: rollback_data_save_requested: false 06-11 17:08:46.914 1356 1356 I update_engine: write_verity: true 06-11 17:08:46.914 1356 1356 I update_engine: Payload: 0 06-11 17:08:46.914 1356 1356 I update_engine: urls: () 06-11 17:08:46.914 1356 1356 I update_engine: size: 1286948403 06-11 17:08:46.914 1356 1356 I update_engine: metadata_size: 94417 06-11 17:08:46.914 1356 1356 I update_engine: metadata_signature: 06-11 17:08:46.914 1356 1356 I update_engine: hash: B4F09617B4E7B8E37799BE83DDBB9CC96F8567A3AA4E2EF7721A9DD473987A3C 06-11 17:08:46.914 1356 1356 I update_engine: type: unknown 06-11 17:08:46.914 1356 1356 I update_engine: fingerprint: 06-11 17:08:46.914 1356 1356 I update_engine: app_id: 06-11 17:08:46.914 1356 1356 I update_engine: already_applied: false 06-11 17:08:46.915 1356 1356 I update_engine: [INFO:postinstall_runner_action.cc(95)] postinstall mount point: /postinstall 06-11 17:08:46.916 1356 1356 I update_engine: [INFO:metrics_utils.cc(318)] Number of Reboots during current update attempt = 0 06-11 17:08:46.917 1356 1356 I update_engine: [INFO:metrics_utils.cc(326)] Payload Attempt Number = 1 06-11 17:08:46.919 1356 1356 I update_engine: [INFO:metrics_utils.cc(343)] Update Monotonic Timestamp Start = 1/1/1970 0:04:33 GMT 06-11 17:08:46.920 1356 1356 I update_engine: [INFO:metrics_utils.cc(352)] Update Boot Timestamp Start = 1/1/1970 0:04:33 GMT 06-11 17:08:46.921 1356 1356 I update_engine: [INFO:update_attempter_android.cc(840)] Clearing update complete marker. 06-11 17:08:46.922 1356 1356 I update_engine: [INFO:update_attempter_android.cc(720)] Scheduling an action processor start. 06-11 17:08:46.923 1356 1356 I update_engine: [INFO:action_processor.cc(51)] ActionProcessor: starting UpdateBootFlagsAction 06-11 17:08:46.924 1356 1356 I update_engine: [INFO:update_boot_flags_action.cc(45)] Marking booted slot as good. 06-11 17:08:46.944 1356 1356 I update_engine: [INFO:action_processor.cc(116)] ActionProcessor: finished UpdateBootFlagsAction with code ErrorCode::kSuccess 06-11 17:08:46.945 1356 1356 I update_engine: [INFO:action_processor.cc(143)] ActionProcessor: starting CleanupPreviousUpdateAction 06-11 17:08:46.946 1356 1356 I update_engine: [INFO:cleanup_previous_update_action.cc(149)] Starting/resuming CleanupPreviousUpdateAction 06-11 17:08:46.947 1356 1356 I update_engine: [INFO:cleanup_previous_update_action.cc(189)] Boot completed, waiting on markBootSuccessful() 06-11 17:08:46.948 1356 1356 I update_engine: EnsureMetadataMounted does nothing in Android mode. 06-11 17:08:46.950 1356 1356 I update_engine: Read merge statistics file failed: No such file or directory 06-11 17:08:46.951 1356 1356 I update_engine: [INFO:cleanup_previous_update_action.cc(261)] Waiting for any previous merge request to complete. This can take up to several minutes. 06-11 17:08:46.952 1356 1356 E update_engine: Read state file failed: No such file or directory 06-11 17:08:46.953 1356 1356 E update_engine: Read state file failed: No such file or directory 06-11 17:08:46.953 1356 1356 E update_engine: Read state file failed: No such file or directory 06-11 17:08:46.954 1356 1356 I update_engine: CheckMergeState for snapshots returned: 0 06-11 17:08:46.955 1356 1356 I update_engine: ProcessUpdateState handling state: 0 06-11 17:08:46.956 1356 1356 I update_engine: [INFO:cleanup_previous_update_action.cc(297)] Can't find any snapshot to merge. 06-11 17:08:46.957 1356 1356 E update_engine: Read state file failed: No such file or directory 06-11 17:08:46.958 1356 1356 I update_engine: [INFO:cleanup_previous_update_action.cc(130)] Stopping/suspending/completing CleanupPreviousUpdateAction 06-11 17:08:46.959 1356 1356 I update_engine: [INFO:cleanup_previous_update_action.cc(479)] Not reporting merge stats because state is None 06-11 17:08:46.960 1356 1356 I update_engine: [INFO:cleanup_previous_update_action.cc(130)] Stopping/suspending/completing CleanupPreviousUpdateAction 06-11 17:08:46.961 1356 1356 I update_engine: [INFO:action_processor.cc(116)] ActionProcessor: finished CleanupPreviousUpdateAction with code ErrorCode::kSuccess 06-11 17:08:46.963 1356 1356 I update_engine: [INFO:action_processor.cc(143)] ActionProcessor: starting InstallPlanAction 06-11 17:08:46.964 1356 1356 I update_engine: [INFO:action_processor.cc(116)] ActionProcessor: finished InstallPlanAction with code ErrorCode::kSuccess 06-11 17:08:46.965 1356 1356 I update_engine: [INFO:action_processor.cc(143)] ActionProcessor: starting DownloadAction 06-11 17:08:46.966 1356 1356 I update_engine: [INFO:install_plan.cc(80)] InstallPlan: 06-11 17:08:46.966 1356 1356 I update_engine: type: new_update 06-11 17:08:46.966 1356 1356 I update_engine: version: 06-11 17:08:46.966 1356 1356 I update_engine: source_slot: A 06-11 17:08:46.966 1356 1356 I update_engine: target_slot: B 06-11 17:08:46.966 1356 1356 I update_engine: initial url: file:///sdcard/Download/omni-13-20230611-zenfone8-MICROG.zip 06-11 17:08:46.966 1356 1356 I update_engine: hash_checks_mandatory: true 06-11 17:08:46.966 1356 1356 I update_engine: powerwash_required: false 06-11 17:08:46.966 1356 1356 I update_engine: switch_slot_on_reboot: true 06-11 17:08:46.966 1356 1356 I update_engine: run_post_install: true 06-11 17:08:46.966 1356 1356 I update_engine: is_rollback: false 06-11 17:08:46.966 1356 1356 I update_engine: rollback_data_save_requested: false 06-11 17:08:46.966 1356 1356 I update_engine: write_verity: true 06-11 17:08:46.966 1356 1356 I update_engine: Payload: 0 06-11 17:08:46.966 1356 1356 I update_engine: urls: () 06-11 17:08:46.966 1356 1356 I update_engine: size: 1286948403 06-11 17:08:46.966 1356 1356 I update_engine: metadata_size: 94417 06-11 17:08:46.966 1356 1356 I update_engine: metadata_signature: 06-11 17:08:46.966 1356 1356 I update_engine: hash: B4F09617B4E7B8E37799BE83DDBB9CC96F8567A3AA4E2EF7721A9DD473987A3C 06-11 17:08:46.966 1356 1356 I update_engine: type: unknown 06-11 17:08:46.966 1356 1356 I update_engine: fingerprint: 06-11 17:08:46.966 1356 1356 I update_engine: app_id: 06-11 17:08:46.966 1356 1356 I update_engine: already_applied: false 06-11 17:08:46.967 1356 1356 I update_engine: [INFO:download_action.cc(86)] Marking new slot as unbootable 06-11 17:08:46.985 1356 1356 I update_engine: [INFO:multi_range_http_fetcher.cc(45)] starting first transfer 06-11 17:08:46.986 1356 1356 I update_engine: [INFO:multi_range_http_fetcher.cc(74)] starting transfer of range 3920+1286948403 06-11 17:08:46.989 1356 1356 I update_engine: [INFO:delta_performer.cc(113)] Completed 0/? operations, 16384/1286948403 bytes downloaded (0%), overall progress 0% 06-11 17:08:46.991 1356 1356 I update_engine: [INFO:delta_performer.cc(344)] Manifest size in payload matches expected value from Omaha 06-11 17:08:46.992 1356 1356 I update_engine: [INFO:delta_performer.cc(884)] Verifying using certificates: /system/etc/security/otacerts.zip 06-11 17:08:46.995 1356 1356 I update_engine: [INFO:payload_verifier.cc(102)] signature blob size = 267 06-11 17:08:46.996 1356 1356 I update_engine: [INFO:payload_verifier.cc(118)] Truncating the signature to its unpadded size: 256. 06-11 17:08:47.004 1356 1356 I update_engine: [INFO:payload_verifier.cc(191)] Failed to verify the signature with 1 keys. 06-11 17:08:47.006 1356 1356 E update_engine: [ERROR:payload_verifier.cc(137)] None of the 1 signatures is correct. Expected hash before padding: 06-11 17:08:47.006 1356 1356 I update_engine: [INFO:utils.cc(414)] Logging array of length: 32 06-11 17:08:47.007 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x00000000 : ce ce cc 61 4e 77 30 ae 2e 25 f7 fb 0d 5e 45 45 06-11 17:08:47.008 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x00000010 : 6d 84 c2 1e d5 65 72 55 45 10 20 10 83 5c 76 ba 06-11 17:08:47.009 1356 1356 E update_engine: [ERROR:payload_verifier.cc(140)] But found RSA decrypted hashes: 06-11 17:08:47.010 1356 1356 I update_engine: [INFO:utils.cc(414)] Logging array of length: 256 06-11 17:08:47.011 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x00000000 : 0b 52 c0 e7 5a ea 17 45 fa 16 88 24 d1 49 50 1d 06-11 17:08:47.012 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x00000010 : ac 7b bb 2f ce 3d 6f d5 d1 4d 05 2b 43 20 6b 67 06-11 17:08:47.013 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x00000020 : a3 11 04 81 11 d1 69 de 34 bc 6a 4e 3c 0d e5 4a 06-11 17:08:47.014 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x00000030 : d0 6e b3 e5 fc 69 57 24 b5 42 c3 bd f9 91 f5 dc 06-11 17:08:47.015 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x00000040 : 63 ac 7d 17 65 f7 c9 82 13 a5 7f 49 f7 e4 94 f2 06-11 17:08:47.016 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x00000050 : 13 aa 91 39 a3 c9 7e 66 f2 7b 93 77 84 72 9c 70 06-11 17:08:47.017 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x00000060 : 8d c4 4e bf 32 96 2f a2 14 0d 44 c3 f2 37 c3 28 06-11 17:08:47.018 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x00000070 : 8f 2d fd 46 e1 35 4a 32 ac 6b 41 2f 02 13 ec 30 06-11 17:08:47.019 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x00000080 : dc 01 9c 40 d8 a5 62 1e d1 e7 f9 07 f0 0a a7 dc 06-11 17:08:47.020 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x00000090 : f0 06 1a 63 d0 5e a3 67 d0 03 33 97 04 c2 f6 ac 06-11 17:08:47.021 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x000000a0 : d2 97 4d 0e 95 98 e9 eb 18 38 34 82 34 5e 3f aa 06-11 17:08:47.022 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x000000b0 : a9 3f f0 7e a0 eb 6e 0b bf 09 03 f6 24 85 e3 5f 06-11 17:08:47.023 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x000000c0 : 11 a2 e4 be c6 4d d3 ff f7 4d f2 63 16 e6 d5 e2 06-11 17:08:47.024 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x000000d0 : 69 23 b2 f7 8a 4d bc 6b f8 22 69 78 59 10 ae 4d 06-11 17:08:47.025 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x000000e0 : 79 1d 34 cc 17 65 ce 9d 1a 75 29 dd 74 78 b3 b9 06-11 17:08:47.026 1356 1356 I update_engine: [INFO:utils.cc(431)] 0x000000f0 : 24 b8 db 5b 15 3b 5a ab 39 e1 c7 3d e2 ad 95 2d 06-11 17:08:47.027 1356 1356 E update_engine: [ERROR:payload_metadata.cc(214)] Manifest hash verification failed. 06-11 17:08:47.028 1356 1356 E update_engine: [ERROR:delta_performer.cc(372)] Mandatory metadata signature validation failed 06-11 17:08:47.028 1356 1356 E update_engine: [ERROR:download_action.cc(227)] Error ErrorCode::kDownloadMetadataSignatureMismatch (26) in DeltaPerformer's Write method when processing the received payload -- Terminating processing 06-11 17:08:47.029 1356 1356 I update_engine: [INFO:delta_performer.cc(215)] Discarding 94684 unused downloaded bytes 06-11 17:08:47.031 1356 1356 I update_engine: [INFO:multi_range_http_fetcher.cc(177)] Received transfer terminated. 06-11 17:08:47.031 1356 1356 I update_engine: [INFO:multi_range_http_fetcher.cc(129)] TransferEnded w/ code 200 06-11 17:08:47.032 1356 1356 I update_engine: [INFO:multi_range_http_fetcher.cc(131)] Terminating. 06-11 17:08:47.033 1356 1356 I update_engine: [INFO:action_processor.cc(116)] ActionProcessor: finished DownloadAction with code ErrorCode::kDownloadMetadataSignatureMismatch 06-11 17:08:47.034 1356 1356 I update_engine: [INFO:action_processor.cc(121)] ActionProcessor: Aborting processing due to failure. 06-11 17:08:47.035 1356 1356 I update_engine: [INFO:update_attempter_android.cc(584)] Processing Done. 06-11 17:08:47.038 1356 1356 E update_engine: Read state file failed: No such file or directory 06-11 17:08:47.039 1356 1356 I update_engine: [INFO:metrics_reporter_android.cc(159)] Current update attempt downloads 0 bytes data ASUS_I006D:/ $

So the update_engine refuse to update the OS using the image /sdcard/Download/omni-13-20230611-zenfone8-MICROG.zip because there is no matching certificate for the new OS image in the file /system/etc/security/otacerts.zip of the running OS.

This is quite a useful feature, but not really necessary for my development phone, so I would like to disable it.

That can be done like this on phones wit installed Magisk:

Create a Magisk Module to replace the file /system/etc/security/otacerts.zip with a zip file with certificates from the original OmniROM image and the self-compiled OmiROM image.

To create that file do:

First copy the file /system/etc/security/otacerts.zip from the original OmniROM installed to the directory /sdcard/Download and then to your local workstation via adb pull command.

The file otacerts.zip for the self-compiled OmniROM image for the ASUS Zenfone 8 is in the directory ./out/target/product/zenfone8/system/etc/security in the build tree for the OmniROM.

Extract the certificate testkey.x509.pem from the file otacerts.zip in the build tree, rename it, add it to the file otacerts.zip from the original OmniROM image, and upload the changed file otacerts.zip to the phone:

instructions to create a new otacerts.zip file

# # download the file /system/etc/security/otacerts.zip from the installed OS # [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ adb pull /system/etc/security/otacerts.zip /system/etc/security/otacerts.zip: 1 file pulled, 0 skipped. 0.5 MB/s (2116 bytes in 0.004s) [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ # # list the contents of the file otacerts.zip from the original OmniROM # [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ unzip -t otacerts.zip Archive: otacerts.zip testing: testkey.x509.pem OK No errors detected in compressed data of otacerts.zip. [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ # extract the certificate from the file otacerts.zip from the local build tree for the OmniROM OS # [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ unzip ../OmniROM_13.0/out/target/product/zenfone8/system/etc/security/otacerts.zip Archive: ../OmniROM_13.0/out/target/product/zenfone8/system/etc/security/otacerts.zip inflating: testkey.x509.pem [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ ls -ltr total 8 -rw-r--r--. 1 xtrnaw7 xtrnaw7 1675 Jan 1 2008 testkey.x509.pem -rw-r--r--. 1 xtrnaw7 xtrnaw7 1023 Jun 11 11:17 otacerts.zip [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ # the name of the certificate for both versions of the OmniROM is the same therefor we rename the certificate from the self-compiled OmniROM # [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ mv testkey.x509.pem my_testkey.x509.pem [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ # add the certificate from the self-compiled OmniROM to the file otacerts.zip from the original OmniROM # [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ zip -u otacerts.zip my_testkey.x509.pem adding: my_testkey.x509.pem (deflated 44%) [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ # check the result # [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ unzip -t otacerts.zip Archive: otacerts.zip testing: testkey.x509.pem OK testing: my_testkey.x509.pem OK No errors detected in compressed data of otacerts.zip. [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ # copy the changed file otacerts.zip to the phone # [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ adb push otacerts.zip /sdcard/Download/ otacerts.zip: 1 file pushed, 0 skipped. 21.5 MB/s (2116 bytes in 0.000s) [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $

Now open an adb shell on the phone and create a dummy Magisk Module to replace the file /system/etc/security/otacerts.zip with the new zip file:

Instructions to create the dummy Magisk Module to replace the file otacerts.zip

# # these commands must be done by the user root # ASUS_I006D:/data/develop1 # id uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 ASUS_I006D:/data/develop1 # # create the dummy magisk Module # ASUS_I006D:/data/develop1 # mkdir -p /data/adb/modules/add_certificate/system/etc/security ASUS_I006D:/data/develop1 # cp /sdcard/Download/otacerts.zip /data/adb/modules/add_certificate/system/etc/security ASUS_I006D:/data/develop1 # ASUS_I006D:/data/develop1 # chmod 644 /data/adb/modules/add_certificate/system/etc/security/otacerts.zip ASUS_I006D:/data/develop1 # ASUS_I006D:/data/develop1 # ls -lZtr /system/etc/security/otacerts.zip -rw-r--r-- 1 root root u:object_r:system_file:s0 1023 2009-01-01 01:00 /system/etc/security/otacerts.zip ASUS_I006D:/data/develop1 # ASUS_I006D:/data/develop1 # ls -ltrZ /data/adb/modules/add_certificate/system/etc/security/otacerts.zip -rw-r--r-- 1 root root u:object_r:system_file:s0 2116 2023-06-11 11:21 /data/adb/modules/add_certificate/system/etc/security/otacerts.zip ASUS_I006D:/data/develop1 #

Next reboot the phone to activate the dummy Magisk Module.

Check the result after the reboot:

ASUS_I006D:/ $ unzip -t /system/etc/security/otacerts.zip Archive: /system/etc/security/otacerts.zip testing: testkey.x509.pem OK testing: my_testkey.x509.pem OK No errors detected in compressed data of /system/etc/security/otacerts.zip. ASUS_I006D:/ $

Now you can update the installed original OmniROM image with a self-compiled OmniROM image and vice versa using the local file functionality of the SystemUpdate App from the OmniROM OS as long as the dummy Magisk module is installed.
It's recommended to create a complete Magisk Module for this replacment if used on a regular base.

Notes

The certificate for the OmniROM OS is valid until March 2041:

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $ keytool -printcert -file testkey.x509.pem 2>/dev/null | grep "Valid from:" Valid from: Fri Oct 25 04:40:22 CEST 2013 until: Tue Mar 12 03:40:22 CET 2041 [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/otacerts ] $

Therefor the new zip file with the certificates should work for a while.

This method only works with compatible OS images - e.g. you cannot use it to update the OmniROM image with the original Android image from ASUS or an image with the LineageOS.font>
(well, you can -- but after the update the new OS will not boot ...)

Temporary allow updating another OS image

To only temporary allow the update to another compatible OS via SystemUpdate App you may use a bind mount. bind mounts need root access therefor an installed Magisk or any other tool to get root access is required.

The steps necessary for this temporary solution are:

First copy the file /system/etc/security/otacerts.zip from the OS image to install to the phone, e.g.:

adb push otacerts.zip /sdcard/Download/my_otacerts.zip

Then create a temporary bind mount on the phone for the file /system/etc/security/otacerts.zip :

su - -c mount -o bind /sdcard/Download/my_otacerts.zip /system/etc/security/otacerts.zip

Now the update via SystemUpdate app will work.

Note:

bind mounts are only temporary and will not survive the reboot

Trouble Shooting

The error message "Update failed with error 51" is printed by the SystemUpdate App when the new OS image is older then the installed OS image. I did not yet find a method to disable that test.

The log files of the update_engine are in the directory /data/misc/update_engine_log/

To get a more detailed error message for the errors of the SystemUpdate app use this command in an adb shell:

ASUS_I006D:/ # logcat -d | grep update_engine | grep ERROR06-11 11:30:32.154 1428 1428 E update_engine: [ERROR:delta_performer.cc(1046)] The current OS build timestamp (1686434647) is newer than the maximum timestamp in the manifest (1683475334) 06-11 11:30:32.155 1428 1428 E update_engine: [ERROR:download_action.cc(227)] Error ErrorCode::kPayloadTimestampError (51) in DeltaPerformer's Write method when processing the received payload -- Terminating processing

Disclaimer

This procedure should NOT be performed on phones in productive use -- the procedure is intended for testing and development only.

How to temporary change a file on a read-only filesystem in the Android OS

URL: https://xdaforums.com/t/how-to-temporary-change-a-file-on-a-read-only-filesystem-in-the-android-os.4595429/

How to temporary change a file on a read-only filesystem in the Android OS

I've mentioned this elsewhere in some of my howtos, but I think this feature is important and useful enough to get its own howto.

Nowadays in the current Android OS version a lot filesystems are mounted read-only and remounting the filesystem read-write does not work anymore. Therefor the files in these filesystems can not be changed anymore.
That's pretty good from a safety standpoint, but sometimes a real showstopper.

But fortunately Android is a Linux based system and therefor there is a solution for this problem called bind mounts. bind mounts are some kind of symbolic links that only exist in memory and are therefor also supported for read-only mounted filesystems. Of course, this is a very simple description for this Linux feature but it should be enough here -- for more in depth details I recommend the Linux documentation.
bind mounts are used by Magisk but they can also be used manually without an installed Magisk - the only requirement for using it manually is root access to the phone.

Here is an example to replace the file /system/etc/hosts with a writable file using a bind mount:

Example:

replace the file /system/etc/hosts with a changed version of the file

# # use a filesystem that supports all necessary permissions and attributes for the new file # # Therefor files in /sdcard/<something> can not be used to replace most of the files in the read-only filesystems # # Make sure that the file used to replace the other file is readable by all user that can also read the original file # ASUS_I006D:/ # cp -a /system/etc/hosts /data/local/tmp/my_hosts ASUS_I006D:/ # # add some additional data to the new file # ASUS_I006D:/ # echo "127.0.0.1 www.heise.de" >>/data/local/tmp/my_hosts ASUS_I006D:/ # # check the result # ASUS_I006D:/ # cat /data/local/tmp/my_hosts 127.0.0.1 localhost ::1 ip6-localhost 127.0.0.1 www.heise.de ASUS_I006D:/ # ASUS_I006D:/ # ls -l /system/etc/hosts /data/local/tmp/my_hosts -rw-r--r-- 1 root root 80 2023-06-11 14:57 /data/local/tmp/my_hosts -rw-r--r-- 1 root root 56 2009-01-01 01:00 /system/etc/hosts ASUS_I006D:/ # # "replace" the file /system/etc/hosts now # ASUS_I006D:/ # su - -c mount -o bind /data/local/tmp/my_hosts /system/etc/hosts ASUS_I006D:/ # # check the results # ASUS_I006D:/ # cat /system/etc/hosts 127.0.0.1 localhost ::1 ip6-localhost 127.0.0.1 www.heise.de ASUS_I006D:/ # # add some more data to the file # ASUS_I006D:/ # echo "127.0.0.1 www.spiegle.de" >>/system/etc/hosts ASUS_I006D:/ # # check the result # ASUS_I006D:/ # cat /system/etc/hosts 127.0.0.1 localhost ::1 ip6-localhost 127.0.0.1 www.heise.de 127.0.0.1 www.spiegle.de ASUS_I006D:/ #

bind mounts are only possible for existing files - bind mounts for non-existent files will fail, e.g:

Example for a bind mount for a non-existent file

ASUS_I006D:/ $ ls /system/bin/bash ls: /system/bin/bash: No such file or directory 1|ASUS_I006D:/ $ 1|ASUS_I006D:/ $ su - -c mount -o bind /system/bin/sh /system/bin/bash mount: '/system/bin/sh'->'/system/bin/bash': No such file or directory 1|ASUS_I006D:/ $

bind mounts are also allowed for directories therefor to "create" a new file just copy the directory, create the new file in that new directory, and then do a bind mount for the existing directory.

Example:

"create" the executable "bash" by copying the executable "sh"

# "create" the executable "bash" by copying the executable "sh" # # This is only an example that should work on every phone! In real life you should copy a real bash executable to the new directory # # # check for a bash executable in the PATH # ASUS_I006D:/ $ which bash 1|ASUS_I006D:/ $ # -> bash does currently not exist on the phone # create a new directory to be used for the bind mount /system/xbin # # Note that this directory must be on a filesystem supporting the standard linux file permissions therefor a directory in /sdcard/<something> is not possible here # And again the directory used must be readable by all uesrs. # ASUS_I006D:/ $ mkdir /data/local/tmp/xbinASUS_I006D:/ $ # copy all files from /system/xbin to the new directory # ASUS_I006D:/ $ su - -c cp -a -r /system/xbin/* /data/local/tmp/xbin ASUS_I006D:/ $ ASUS_I006D:/ $ ls -l /data/local/tmp/xbin total 1772 lrwxrwxrwx 1 root shell 3 2009-01-01 01:00 vi -> vim -rwxr-xr-x 1 root shell 1813848 2009-01-01 01:00 vim ASUS_I006D:/ $ # create fake "bash" executable in the new directory # ASUS_I006D:/ $ cp /system/bin/sh /data/local/tmp/xbin/bashASUS_I006D:/ $ ASUS_I006D:/ $ ls -l /data/local/tmp/xbintotal 2080 -rwxr-xr-x 1 shell shell 312024 2023-06-11 13:56 bash lrwxrwxrwx 1 root shell 3 2009-01-01 01:00 vi -> vim -rwxr-xr-x 1 root shell 1813848 2009-01-01 01:00 vim ASUS_I006D:/ $ # bind mount the directory /system/xbin on the new directory # ASUS_I006D:/ $ su - -c mount -o bind /data/local/tmp/xbin /system/xbin ASUS_I006D:/ $ # and now there is "bash" executable in /system/xbin available # ASUS_I006D:/ $ ls -l /system/xbin total 2080 -rwxr-xr-x 1 shell shell 312024 2023-06-11 13:56 bash lrwxrwxrwx 1 root shell 3 2009-01-01 01:00 vi -> vim -rwxr-xr-x 1 root shell 1813848 2009-01-01 01:00 vim ASUS_I006D:/ $ which bash /system/xbin/bash # You can now even copy additional files to the bind mounteddirectory, e.g.: ASUS_I006D:/ $ cp /sdcard/Download/zip /system/xbinASUS_I006D:/ $ ASUS_I006D:/ $ chmod 755 /system/xbin/zip ASUS_I006D:/ $ ASUS_I006D:/ $ ls -ltr /system/xbin total 2576 -rwxr-xr-x 1 root shell 1813848 2009-01-01 01:00 vim lrwxrwxrwx 1 root shell 3 2009-01-01 01:00 vi -> vim -rwxr-xr-x 1 shell shell 312024 2023-06-11 13:56 bash -rwxr-xr-x 1 shell shell 503992 2023-06-11 14:03 zip ASUS_I006D:/ $ ASUS_I006D:/ $ zip -h Copyright (c) 1990-2008 Info-ZIP - Type 'zip "-L"' for software license. Zip 3.0 (July 5th 2008). Usage: zip [-options] [-b path] [-t mmddyyyy] [-n suffixes] [zipfile list] [-xi list] The default action is to add or replace zipfile entries from list, which can include the special name - to compress standard input. If zipfile and list are omitted, zip compresses stdin to stdout. -f freshen: only changed files -u update: only changed or new files -d delete entries in zipfile -m move into zipfile (delete OS files) -r recurse into directories -j junk (don't record) directory names -0 store only -l convert LF to CR LF (-ll CR LF to LF) -1 compress faster -9 compress better -q quiet operation -v verbose operation/print version info -c add one-line comments -z add zipfile comment -@ read names from stdin -o make zipfile as old as latest entry -x exclude the following names -i include only the following names -F fix zipfile (-FF try harder) -D do not add directory entries -A adjust self-extracting exe -J junk zipfile prefix (unzipsfx) -T test zipfile integrity -X eXclude eXtra file attributes -y store symbolic links as the link instead of the referenced file -e encrypt -n don't compress these suffixes -h2 show more help

Hints

bind mounts are supported by the filesystems ext4, vfat, f2fs. and most probably a lot of other filesystems used on machines running Android.

Be careful when replacing executables, especially when replacing an important file like /system/bin/sh. And you should never replace a file that is just a symbolic link for toybox or busybox.

bind mounts created while the OS is running are only useful for files that are dynamically read by the OS. It does not make sense to replace for example one of the *.rc files using a bind mount because these files are only read once while booting the OS.

The filesystems used for source and target for a bind mount should support the same set of attributes and permissions or the result may not be like expected. E.g. you can not bind mount an executable with a file in the directory /sdcard/<something> because that filesystem for /sdcard does not support the executable permission:

Example for bind mount an executable in /sdcard/Download

ASUS_I006D:/ $ ls -l /bin/ziptool -rwxr-xr-x 1 root shell 28688 2009-01-01 01:00 /bin/ziptool ASUS_I006D:/ $ ASUS_I006D:/ $ cp -a /bin/ziptool /sdcard/Download/ ASUS_I006D:/ $ ASUS_I006D:/ $ chmod 755 /sdcard/Download/ziptool ASUS_I006D:/ $ ASUS_I006D:/ $ ls -l /bin/ziptool /sdcard/Download/ziptool -rwxr-xr-x 1 root shell 28688 2009-01-01 01:00 /bin/ziptool -rw-rw---- 1 u0_a121 media_rw 28688 2023-06-11 18:09 /sdcard/Download/ziptool ASUS_I006D:/ $ ASUS_I006D:/ $ su - -c mount -o bind /sdcard/Download/ziptool /bin/ziptool ASUS_I006D:/ $ ASUS_I006D:/ $ ziptool /system/bin/sh: ziptool: can't execute: Permission denied 126|ASUS_I006D:/ $ /bin/ziptool /system/bin/sh: /bin/ziptool: can't execute: Permission denied 126|ASUS_I006D:/ $ ls -l /bin/ziptool -rw-rw---- 1 u0_a121 media_rw 28688 2023-06-11 18:09 /bin/ziptool ASUS_I006D:/ $

The source for the bind mount must be in a directory that is readable for all user that use the bind mounted file, e.g.

Example for bind mount an executable in a directory not accessable by all user

ASUS_I006D:/ $ su - -c cp -a /bin/ziptool /data/adb/ziptool ASUS_I006D:/ $ ASUS_I006D:/ $ su - -c ls -l /data/adb/ziptool -rwxr-xr-x 1 root shell 28688 2009-01-01 01:00 /data/adb/ziptool ASUS_I006D:/ $ ASUS_I006D:/ $ su - -c mount -o bind /data/adb/ziptool /bin/ziptool ASUS_I006D:/ $ ASUS_I006D:/ $ ziptool /system/bin/sh: ziptool: inaccessible or not found 127|ASUS_I006D:/ $ 127|ASUS_I006D:/ $ file /bin/ziptool /bin/ziptool: cannot open: Permission denied ASUS_I006D:/ $

To remove a bind mount just do a umount, e.g:

Example for remove a bind mount

# # remove the bind mount # ASUS_I006D:/ $ su - -c umount /bin/ziptool ASUS_I006D:/ $ ASUS_I006D:/ $ which ziptool /system/bin/ziptool ASUS_I006D:/ $ ziptool ziptool: run as ziptool with unzip or zipinfo as the first argument, or symlink 1|ASUS_I006D:/ $

Another method to remove all bind mounts is a reboot of the phone

It's recommended to use a sub directory in the directory /data/local/tmp for executables used in bind mounts, example:

Example for bind mount an executable in /data/local/tmp/

ASUS_I006D:/ $ mkdir /data/local/tmp/bind_mounts ASUS_I006D:/ $ ASUS_I006D:/ $ cp -a /bin/ziptool /data/local/tmp/bind_mounts ASUS_I006D:/ $ ASUS_I006D:/ $ ls -l /data/local/tmp/bind_mounts/ziptool -rwxr-xr-x 1 shell shell 28688 2009-01-01 01:00 /data/local/tmp/bind_mounts/ziptool ASUS_I006D:/ $ ASUS_I006D:/ $ su - -c mount -o bind /data/local/tmp/bind_mounts/ziptool /bin/ziptool ASUS_I006D:/ $ ASUS_I006D:/ $ ziptool ziptool: run as ziptool with unzip or zipinfo as the first argument, or symlink 1|ASUS_I006D:/ $

bind mounts are also useful for writable files to be able to revert the changes to a file. Example:

Example for using bind mount for a writable file

ASUS_I006D:/ $ cat /data/local/tmp/testfile.txt This is a test file ASUS_I006D:/ $ # # create a copy of the writable file # ASUS_I006D:/ $ cat /data/local/tmp/testfile_for_tests.txt This is another test file # # bind mount the file with the copy of the file # ASUS_I006D:/ $ su - -c mount -o bind /data/local/tmp/testfile_for_tests.txt /data/local/tmp/testfile.txt ASUS_I006D:/ $ # # check the result # ASUS_I006D:/ $ cat /data/local/tmp/testfile.txt This is another test file ASUS_I006D:/ $ # # add some data to the bind mounted file now # ASUS_I006D:/ $ echo "This text will not change the original file" >> /data/local/tmp/testfile.txtASUS_I006D:/ $ ASUS_I006D:/ $ cat /data/local/tmp/testfile.txtThis is another test file This text will not change the original file ASUS_I006D:/ $ # # remove the bind mount to restore the original file contents # ASUS_I006D:/ $ su - -c umount /data/local/tmp/testfile.txt ASUS_I006D:/ $ ASUS_I006D:/ $ cat /data/local/tmp/testfile.txt This is a test file ASUS_I006D:/ $

Of course, this can also be done by creating a backup copy of the file and restoring the backup copy if necessary. But if the changes to the file make the phone unusable or cause it to crash, it is more difficult to restore the file. If you use a bind mount, the file will be restored "automatically" at the next reboot.

Use the mount command to check if a file is bind mounted, e.g.

Check if a file is bind mounted

ASUS_I006D:/ # mount | grep /system/etc/hosts1|ASUS_I006D:/ # 1|ASUS_I006D:/ # mount -o bind /data/local/tmp/hosts /system/etc/hostsASUS_I006D:/ # ASUS_I006D:/ # mount | grep /system/etc/hosts/dev/block/dm-33 on /system/etc/hosts type f2fs (rw,lazytime,seclabel,nosuid,nodev,noatime,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,extent_cache,mode=adaptive,active_logs=6,reserve_root=32768,resuid=0,resgid=1065,inlinecrypt,alloc_mode=default,fsync_mode=nobarrier) ASUS_I006D:/ #

Notes

see the forum entry How to upgrade the OS with another Android "distribution" for an example usage for this method

see the forum entry How To change any file or directory using Magisk for how to use this method in a Magisk module for persisent changes

see the forum entry How to make files in /system writable for another exampe how to use this feature in Magisk

Up to version 25.x of Magisk (at least in Magisk v24.x and v25.x) Magisk mounted the /system/bin directory read-write, so that temporary changes for the files in /system/bin were possible. But because of the more or less not existent free space in the filesystem used for /system the use of this feature was very limited. And this feature does not exist in Magisk v26.x or newer anymore.
(see also this forum message: https://xdaforums.com/t/magisk-general-support-discussion.3432382/page-2815#post-88617909 )

How to change the kernel boot parameter for the Android kernel

URL: https://xdaforums.com/t/how-to-change-the-kernel-boot-parameter-for-the-android-kernel.4600821/

How to change the kernel boot parameter for the Android kernel

While trying to get TWRP to boot on a phone with LIneageOS 20.x installed, I found some information about the boot parameters for the Android OS, which I document with this forum post.

To view the current kernel parameter for the Android kernel the command

cat /proc/cmdline

can be used. e.g

1|ASUS_I006D:/ $ su - -c cat /proc/cmdlinecgroup_disable=pressure log_buf_len=256K earlycon=msm_geni_serial,0x98c000 rcupdate.rcu_expedited=1 rcu_nocbs=0-7 kpti=off noirqdebug androidboot.console=ttyMSM0 androidboot.hardware=qcom androidboot.memcg=1 androidboot.usbcontroller=a600000.dwc3 cgroup.memory=nokmem,nosocket console=ttyMSM0,115200n8 ip6table_raw.raw_before_defrag=1 iptable_raw.raw_before_defrag=1 loop.max_part=7 lpm_levels.sleep_disabled=1 msm_rtb.filter=0x237 pcie_ports=compat service_locator.enable=1 swiotlb=0 kpti=off buildvariant=userdebug androidboot.verifiedbootstate=orange androidboot.keymaster=1 androidboot.bootdevice=1d84000.ufshc androidboot.fstab_suffix=default androidboot.boot_devices=soc/1d84000.ufshc androidboot.serialno=M6AIB760D0939LX androidboot.baseband=msm msm_drm.dsi_display0=qcom,mdss_dsi_samsung_fhd_cmd: androidboot.slot_suffix=_b rootwait ro init=/init androidboot.dtbo_idx=0 androidboot.dtb_idx=0 androidboot.force_normal_boot=1 androidboot.pre-ftm=0 androidboot.id.prj=4 androidboot.id.stage=7 androidboot.id.sku=3 androidboot.id.rf=1 androidboot.id.pcb=4731 androidboot.id.nfc=1 androidboot.country_code=EU androidboot.bootcount=0 androidboot.rawdump_en=0 androidboot.asus.authorized=0 androidboot.cpuid.hash=551c6708e291a3a57e013f5bdf2afe4a androidboot.toolid=15715839e737449088fe70f7faf618f0 SB=Y androidboot.fused=1 SBNR=Y androidboot.fused.norpmb=1 androidboot.id.panel=1 androidboot.id.ufs=3 androidboot.factory.crc=0x4FE99984 androidboot.ddr.manufacturer_id=FF androidboot.ddr.device_type=8 LCD=AC0E0041 androidboot.bootreason=hard ASUS_I006D:/ $

Additional boot parameter can be defined in the boot partition. To do this the Magisk binary magiskboot can be used; Example (for doing this in an Android shell):
# # the commands must be done as user root # ASUS_I006D:/ $ su - ASUS_I006D:/ # mkdir /data/develop ASUS_I006D:/ # cd /data/develop ASUS_I006D:/data/develop # getprop ro.boot.slot_suffix _b ASUS_I006D:/data/develop # # # unpack the current boot partition to the current directory # # Note: The parameter -h for magiskboot is required to write the header file # ASUS_I006D:/data/develop # /data/adb/magisk/magiskboot unpack -h /dev/block/by-name/boot_bParsing boot image: [/dev/block/by-name/boot_b] HEADER_VER [4] KERNEL_SZ [39909888] RAMDISK_SZ [43610592] OS_VERSION [13.0.0] OS_PATCH_LEVEL [2023-05] PAGESIZE [4096] CMDLINE [] KERNEL_FMT [raw] RAMDISK_FMT [lz4_legacy] VBMETA ASUS_I006D:/data/develop # ASUS_I006D:/data/develop # ls -l total 129960 -rw-rw-rw- 1 root root 50 2023-06-25 14:31 header -rw-r--r-- 1 root root 39909888 2023-06-25 14:31 kernel -rw-r--r-- 1 root root 93031052 2023-06-25 14:31 ramdisk.cpio ASUS_I006D:/data/develop # ASUS_I006D:/data/develop # cat header cmdline= os_version=13.0.0 os_patch_level=2023-05 ASUS_I006D:/data/develop #

To define additional boot parameter add them to the line cmdline= in the file header and repack the boot image, e.g:

ASUS_I006D:/data/develop # sed -i -e "s/cmdline=/cmdline=debug /g" headerASUS_I006D:/data/develop # ASUS_I006D:/data/develop # cat header cmdline=debug os_version=13.0.0 os_patch_level=2023-05 ASUS_I006D:/data/develop # ASUS_I006D:/data/develop # /data/adb/magisk/magiskboot repack /dev/block/by-name/boot_b ../new_boot.imgParsing boot image: [/dev/block/by-name/boot_b] HEADER_VER [4] KERNEL_SZ [39909888] RAMDISK_SZ [43610592] OS_VERSION [13.0.0] OS_PATCH_LEVEL [2023-05] PAGESIZE [4096] CMDLINE [] KERNEL_FMT [raw] RAMDISK_FMT [lz4_legacy] VBMETA Repack to boot image: [../new_boot.img] HEADER_VER [4] KERNEL_SZ [39909888] RAMDISK_SZ [43610592] OS_VERSION [13.0.0] OS_PATCH_LEVEL [2023-05] PAGESIZE [4096] CMDLINE [debug] ASUS_I006D:/data/develop #

Use dd (or fastboot) to flash the new boot image, e.g:

dd if=../new_boot.img of=/dev/block/by-name/boot_b

Notes:

To define additional kernel parameter add them to the line using the blank as separator for the parameter - e.g.:

cmdline=loglevel=7 debug

The list of known kernel parameter for the ASUS Zenfone 8 can be found in the file

./kernel/asus/sm8350/Documentation/admin-guide/kernel-parameters.txt

in the source for the Android OS used (but most of the parameter described there are useless for the Android OS I guess ...)

The method described above can also be used to define additional kernel parameter for the kernel in the TWRP image. Note that you should never delete the kernel parameter twrpfastboot=1 in the TWRP boot image.

Unpacking and repacking the boot image can also be done on a PC running the Linux OS using the x86 version of magiskboot from the Magisk apk file.

To extract the magiskboot binary for 64 Bit Linux from the apk file use these commands:

unzip -p Magisk-v26.1.apk lib/x86_64/libmagiskboot.so >magiskboot chmod 755 magiskboot
The x86 binaries from the Magisk apk file are static linked binaries and should work on every Linux OS running on x86.

The Magisk Module cmdpatch (https://github.com/Magisk-Modules-Alt-Repo/cmdpatch) contains a script to add new kernel parameter

Update 30.06.2023

Infos from additional posts in the thread https://xdaforums.com/t/how-to-change-the-kernel-boot-parameter-for-the-android-kernel.4600821/ :

"There's no particular need to unpack/pack the image.
The image added options to cmdline are right there in the header.
Just hexedit at offset 0x40 (for Android image versions 0, 1, 2) or offset 0x2c (for Android image versions 3, 4).
If your device uses AVB0 you may need to regenerate that."
ImgUtil is a Win32 utility for modifying Android boot images. (It was formerly named "bootutil.exe".)

How to use TWRP if LineageOS 20.x is installed

URL: https://xdaforums.com/t/how-to-use-twrp-if-lineageos-20-x-is-installed.4599721/

How to use TWRP if LineageOS 20.x is installed

After installing LineageOS 20.x or newer on a phone, booting the phone from the TWRP image using the command

sudo fastboot boot <twrp_image_file>

does not work anymore : The phone starts to boot; prints the ASUS logo and hangs forever (at least on the ASUS Zenfone 8).

To work around this annoying problem, it is necessary to create a new TWRP boot image file using the boot image from the LineageOS and the ramdisk and kernel parameter from the TWRP image.

This can be done as follows:

Download the image file for the boot partition (boot.img) used for LineageOS 20.x for your phone

For the ASUS Zenfone 8 the file boot.img can be downloaded from this URL: https://download.lineageos.org/devices/sake/builds:

wget https://mirrorbits.lineageos.org/full/sake/20230616/boot.img

Unpack the boot image file using the Magisk binary magiskboot:

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos ] $ ls -ltr total 98308 -rw-rw-r--. 1 xtrnaw7 xtrnaw7 100663296 Jun 16 11:19 boot.img [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos ] $ # the parameter -h for magiskboot is necessary to create the file header # [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos ] $ magiskboot unpack -h boot.img Parsing boot image: [boot.img] HEADER_VER      [4] KERNEL_SZ       [39909888] RAMDISK_SZ      [1663229] OS_VERSION      [13.0.0] OS_PATCH_LEVEL [2023-05] PAGESIZE        [4096] CMDLINE         [] KERNEL_FMT      [raw] RAMDISK_FMT     [lz4_legacy] VBMETA [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos ] $ [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos ] $ ls -ltr total 140364 -rw-rw-r--. 1 xtrnaw7 xtrnaw7 100663296 Jun 16 11:19 boot.img -rw-r--r--. 1 xtrnaw7 xtrnaw7 39909888 Jun 25 19:00 kernel -rw-rw-rw-. 1 xtrnaw7 xtrnaw7        50 Jun 25 19:00 header -rw-r--r--. 1 xtrnaw7 xtrnaw7   3147776 Jun 25 19:00 ramdisk.cpio [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos ] $

In another directory unpack the TWRP image for the phone using magiskboot:

# the parameter -h for magiskboot is necessary to create the file header # [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos/twrp ] $ magiskboot unpack -h ../../twrp-3.7.0_12-0-I006D-enhanced.img Parsing boot image: [../../twrp-3.7.0_12-0-I006D-enhanced.img] HEADER_VER      [3] KERNEL_SZ       [18618540] RAMDISK_SZ      [38299974] OS_VERSION      [99.87.36] OS_PATCH_LEVEL [2099-12] PAGESIZE        [4096] CMDLINE         [twrpfastboot=1] KERNEL_FMT      [gzip] RAMDISK_FMT     [gzip] VBMETA [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos ] $ [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos/twrp ] $ ls -ltr total 133992 -rw-rw-rw-. 1 xtrnaw7 xtrnaw7       66 Jun 25 19:03 header -rw-r--r--. 1 xtrnaw7 xtrnaw7 41943552 Jun 25 19:03 kernel -rw-r--r--. 1 xtrnaw7 xtrnaw7 95251796 Jun 25 19:03 ramdisk.cpio [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos/twrp ] $

Now copy the file ramdisk.cpio file from the TWRP image to the directory with the unpacked boot image file from the LineageOS :

# backup the existing ramdisk file # cp /data/develop/android/twrp_for_lineageos/ramdisk.cpio /data/develop/android/twrp_for_lineageos/ramdisk.cpio.org # copy the ramdisk file from the TWRP image # cp /data/develop/android/twrp_for_lineageos/twrp/ramdisk.cpio /data/develop/android/twrp_for_lineageos/ramdisk.cpio

Then copy the kernel parameter from the TWRP image to the file header from the LineageOS boot image file:

# print the kernel parameter from the TWRP image # [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos/twrp ] $ grep cmd /data/develop/android/twrp_for_lineageos/twrp/header cmdline=twrpfastboot=1 # backup the file header from the LineageOS boot image file # [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos/twrp ] $ cp /data/develop/android/twrp_for_lineageos/header /data/develop/android/twrp_for_lineageos/header.org # add the kernel parameter from the TWRP image to the file header from the LineageOS # [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos ] $ sed -i -e "s/cmdline=/cmdline=twrpfastboot=1/g" /data/develop/android/twrp_for_lineageos/header [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos ] $ # check the result # [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos/twrp ] $ cat /data/develop/android/twrp_for_lineageos/header cmdline=twrpfastboot=1 os_version=13.0.0 os_patch_level=2023-05 [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos/twrp ]

Now repack the boot image file from the LineageOS 20.x to create the new TWRP image:

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos ] $ magiskboot repack ./boot.img ./twrp_3.7.0_12-0-I006D_for_lineageOS20.img Parsing boot image: [./boot.img] HEADER_VER      [4] KERNEL_SZ       [39909888] RAMDISK_SZ      [1663229] OS_VERSION      [13.0.0] OS_PATCH_LEVEL [2023-05] PAGESIZE        [4096] CMDLINE         [] KERNEL_FMT      [raw] RAMDISK_FMT     [lz4_legacy] VBMETA Repack to boot image: [./twrp_3.7.0_12-0-I006D_for_lineageOS20.img] HEADER_VER      [4] KERNEL_SZ       [39909888] RAMDISK_SZ      [44166258] OS_VERSION      [13.0.0] OS_PATCH_LEVEL [2023-05] PAGESIZE        [4096] CMDLINE         [twrpfastboot=1] [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos ] $ [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos ] $ ls -ltr twrp_3.7.0_12-0-I006D_for_lineageOS20.img -rw-r--r--. 1 xtrnaw7 xtrnaw7 100663296 Jun 25 19:11 twrp_3.7.0_12-0-I006D_for_lineageOS20.img [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/twrp_for_lineageos ] $

The TWRP image file twrp_3.7.0_12-0-I006D_for_lineageOS20.img can now be used boot the phone via fastboot if the LineageOS 20.x is installed.

I successfuly tested this on an ASUS Zenfone 8 with the Lineage OS 20.x from the file lineage-20.0-20230616-nightly-sake-signed.zip and the TWRP image file twrp-3.7.0_12-0-I006D-enhanced.img.

Update 26.04.2024

I also tested this approach successfully with the in-official LineageOS 21.x image lineage-21.0-20240220-UNOFFICIAL-KSU-GPU_UV-sake.zip and the TWRP image file twrp-3.7.0_12-0-I006D-enhanced.img.

Update 25.06.2024

I also tested this approach successfully with the StatixOS and LMODroid

Note that the TWRP image for LineageOS created using this method is version and build (!) dependent : A TWRP image must be created for each LineageOS image (even with the same LIneageOS version).
And the TWRP images for LineageOS cannot be used if another ROM is installed on the phone.

Therefor this is really only a very ugly workaround!

Update 26.06.2024

see also How to create a TWRP image for LineageOS based ROMs using a script

Update 07.09.2024

see How to access the system partitions while the phone is booted into the LineageOS recovery for how to access the system partitions if the phone is booted into the original recovery from the LineageOS

Trouble Shoooting

Problem: The TWRP GUI does not resume after the screensaver from TWRP was active (-> the screen remains black; the access to TWRP via adb shell still works)

Workaround

increase the screen timeout in the TWRP GUI or disable the screen saver in the TWRP GUI
(The TWRP settings are stored in the config file /data/media/0/TWRP/.twrps; this file is a binary file that can not be edited)

I did not find any other issues until now .. but I only did a few tests. I guess there are most probably more issues.

But the main function I use TWRP for in this configuration is access via adb - and that works.
(The recovery from the LineageOS also supports access via adb but adb is disabled after booting into the recovery and must be manually enabled in the GUI of the recovery)

Problem: The twrp executable does not work

Booting the phone from the twrp image file works but executing the twrp binary does not work and print an error message like this:
I006D:/ # twrp sideload TWRP does not appear to be running. Waiting for TWRP to start . . . Press CTRL + C to quit. ^C 130|I006D:/ # ^D

Solution

The TWRP image used does not match the installed OS on the phone. Create and use a TWRP version for the OS installed on the phone to fix this error.

Notes

Booting the phone from the recovery image for LineageOS 19.x does also not work anymore after installing LineageOS 20.x.

The file boot.img from the Download page for LineageOS 20.x (https://download.lineageos.org/devices/sake/builds for the ASUS Zenfone 8) is also the Recovery image for that OS.
Using that the file boot.img to boot the phone via "sudo fastboot boot boot.img" does not work

Unpacking and repacking the boot image files in the example above was done on a PC running the Linux OS and using the x86 version of magiskboot from the Magisk apk file.

To extract the magiskboot binary for 64 Bit Linux from the apk file use these commands:

unzip -p Magisk-v26.1.apk lib/x86_64/libmagiskboot.so >magiskboot chmod 755 magiskboot
The x86 binaries from the Magisk apk file are static linked binaries and should work on every Linux OS running on x86.

The commands listed above can also be done in an adb session on the phone with installed Magisk or in a booted TWRP image if no machine running Linux is available.

ASUS Zenfone 8 Firmware Download

URL: https://xdaforums.com/t/zenfone-8-need-android-12-firmware-to-downgrade-from-android-13.4617265/#post-88887299

As of 17.08.2023 the firmware for the ASUS Zenfone 8 can not be downloaded from the ASUS web page anymore.

Update 24.09.2023 /bs

There is now a forum entry with a collection of links to available OS images for the ASUS Zenfone 8:

https://xdaforums.com/t/zenfone-8-firmware-ota-collection.4620171/

ASUS Zenfone 8 Firmware download

There are some firmware images for the ASUS Zenfone 8 on this page:

https://www.firmware27.info/2023/03/asus-zenfone-8-zs590ks-flash-file.html

There is a link to the official ASUS download page and a link to a copy of the image on google drive for each firmware image.
The links to the ASUS download page do not work anymore but at least some of the links to the google drive work.

I successfully downloaded the image Asus ZenFone 8 (ZS590KS) Raw Firmware (last entry) from the link to google drive

The file looks like a raw image

The checksum of the zip file does not match the checksum of the original zip file with the raw image from ASUS:

Original ZIP file from ASUS

[ OmniRom 13 Dev - xtrnaw7@t15g ~ ] $ cksum /data/backup/ASUS_ZENFONE8/raw_images/*/*MR*2204*zip 2848852431 4006660382 /data/backup/ASUS_ZENFONE8/raw_images/Android12/WW-ZS590KS-31.1010.0410.61-MR0-2204-user-20220422.zip [ OmniRom 13 Dev - xtrnaw7@t15g ~ ] $

ZIP file downloaded from that page

[ OmniRom 13 Dev - xtrnaw7@t15g ~ ] $ cksum /data/download/'[up_addROM.com]_WW-ZS590KS-31.1010.0410.61-MR0-2204-user-20220422(1).zip' 1693297488 4006660447 /data/download/[up_addROM.com]_WW-ZS590KS-31.1010.0410.61-MR0-2204-user-20220422(1).zip [ OmniRom 13 Dev - xtrnaw7@t15g ~ ] $

But the check sums for the zip file contents do match the the check sums for the contents of the original zip file:

cksum */* | tee chksum.out

[ OmniRom 13 Dev - xtrnaw7@t15g ~/test1 ] $ cksum */* | tee chksum.out 1013896308 602112 original/abl.elf 2544193035 38517720 original/adb 1415468060 38320640 original/adb.exe 440910305 97792 original/AdbWinApi.dll 2062945662 62976 original/AdbWinUsbApi.dll 785625886 33554432 original/ADF.img 2047031777 8536296 original/asdf_384M.img 4046808373 10703200 original/ASUS-xrom.img 3346853190 38629376 original/boot-debug-gki.img 1689467234 55615488 original/boot-debug.img 684207671 100663296 original/boot-gki.img 217406089 100663296 original/boot.img 3762944689 1054 original/crc32_checksum.txt 3296223989 1007 original/crc32_img_size.txt 2101368269 8388608 original/dtbo.img 1823614613 14252200 original/fastboot 3212560529 13016576 original/fastboot.exe 3536915896 1705200 original/fastboot_for_mac 4294967295 0 original/firmware 1310895658 13800 original/flashall_AFT.cmd 3109226029 45056 original/gpt_both0.bin 2942924624 45056 original/gpt_both1.bin 3718285888 45056 original/gpt_both2.bin 3224877491 45056 original/gpt_both3.bin 3273120389 45056 original/gpt_both4.bin 2550846493 45056 original/gpt_both5.bin 3119167302 45056 original/gpt_both6.bin 105651610 135680 original/grep.exe 4294967295 0 original/lib64 4066097203 711720 original/make_f2fs 1138700453 413696 original/make_f2fs.exe 2657452820 4234408 original/mke2fs 245949360 4313600 original/mke2fs.exe 94533207 313 original/multiple_update_image.bat 852544587 356 original/multiple_update_image.sh 2645684121 33554432 original/persist.img 1741426147 6872867696 original/super.img 3772440178 14168 original/update_image.bat 3869862274 10484 original/update_image_for_mac.sh 2638570025 11245 original/update_image.sh 4209265207 8192 original/vbmeta.img 3157482219 4096 original/vbmeta_system.img 3413920804 6586368 original/vendor_boot-debug.img 2926086928 100663296 original/vendor_boot.img 1013896308 602112 unknown/abl.elf 2544193035 38517720 unknown/adb 1415468060 38320640 unknown/adb.exe 440910305 97792 unknown/AdbWinApi.dll 2062945662 62976 unknown/AdbWinUsbApi.dll 785625886 33554432 unknown/ADF.img 2047031777 8536296 unknown/asdf_384M.img 4046808373 10703200 unknown/ASUS-xrom.img 3346853190 38629376 unknown/boot-debug-gki.img 1689467234 55615488 unknown/boot-debug.img 684207671 100663296 unknown/boot-gki.img 217406089 100663296 unknown/boot.img 3762944689 1054 unknown/crc32_checksum.txt 3296223989 1007 unknown/crc32_img_size.txt 2101368269 8388608 unknown/dtbo.img 1823614613 14252200 unknown/fastboot 3212560529 13016576 unknown/fastboot.exe 3536915896 1705200 unknown/fastboot_for_mac 4294967295 0 unknown/firmware 1310895658 13800 unknown/flashall_AFT.cmd 3109226029 45056 unknown/gpt_both0.bin 2942924624 45056 unknown/gpt_both1.bin 3718285888 45056 unknown/gpt_both2.bin 3224877491 45056 unknown/gpt_both3.bin 3273120389 45056 unknown/gpt_both4.bin 2550846493 45056 unknown/gpt_both5.bin 3119167302 45056 unknown/gpt_both6.bin 105651610 135680 unknown/grep.exe 4294967295 0 unknown/lib64 4066097203 711720 unknown/make_f2fs 1138700453 413696 unknown/make_f2fs.exe 2657452820 4234408 unknown/mke2fs 245949360 4313600 unknown/mke2fs.exe 94533207 313 unknown/multiple_update_image.bat 852544587 356 unknown/multiple_update_image.sh 2645684121 33554432 unknown/persist.img 1741426147 6872867696 unknown/super.img 3772440178 14168 unknown/update_image.bat 3869862274 10484 unknown/update_image_for_mac.sh 2638570025 11245 unknown/update_image.sh 4209265207 8192 unknown/vbmeta.img 3157482219 4096 unknown/vbmeta_system.img 3413920804 6586368 unknown/vendor_boot-debug.img 2926086928 100663296 unknown/vendor_boot.img [ OmniRom 13 Dev - xtrnaw7@t15g ~/test1

sort chksum.out

[ OmniRom 13 Dev - xtrnaw7@t15g ~/test1 ] $ sort chksum.out 1013896308 602112 original/abl.elf 1013896308 602112 unknown/abl.elf 105651610 135680 original/grep.exe 105651610 135680 unknown/grep.exe 1138700453 413696 original/make_f2fs.exe 1138700453 413696 unknown/make_f2fs.exe 1310895658 13800 original/flashall_AFT.cmd 1310895658 13800 unknown/flashall_AFT.cmd 1415468060 38320640 original/adb.exe 1415468060 38320640 unknown/adb.exe 1689467234 55615488 original/boot-debug.img 1689467234 55615488 unknown/boot-debug.img 1741426147 6872867696 original/super.img 1741426147 6872867696 unknown/super.img 1823614613 14252200 original/fastboot 1823614613 14252200 unknown/fastboot 2047031777 8536296 original/asdf_384M.img 2047031777 8536296 unknown/asdf_384M.img 2062945662 62976 original/AdbWinUsbApi.dll 2062945662 62976 unknown/AdbWinUsbApi.dll 2101368269 8388608 original/dtbo.img 2101368269 8388608 unknown/dtbo.img 217406089 100663296 original/boot.img 217406089 100663296 unknown/boot.img 245949360 4313600 original/mke2fs.exe 245949360 4313600 unknown/mke2fs.exe 2544193035 38517720 original/adb 2544193035 38517720 unknown/adb 2550846493 45056 original/gpt_both5.bin 2550846493 45056 unknown/gpt_both5.bin 2638570025 11245 original/update_image.sh 2638570025 11245 unknown/update_image.sh 2645684121 33554432 original/persist.img 2645684121 33554432 unknown/persist.img 2657452820 4234408 original/mke2fs 2657452820 4234408 unknown/mke2fs 2926086928 100663296 original/vendor_boot.img 2926086928 100663296 unknown/vendor_boot.img 2942924624 45056 original/gpt_both1.bin 2942924624 45056 unknown/gpt_both1.bin 3109226029 45056 original/gpt_both0.bin 3109226029 45056 unknown/gpt_both0.bin 3119167302 45056 original/gpt_both6.bin 3119167302 45056 unknown/gpt_both6.bin 3157482219 4096 original/vbmeta_system.img 3157482219 4096 unknown/vbmeta_system.img 3212560529 13016576 original/fastboot.exe 3212560529 13016576 unknown/fastboot.exe 3224877491 45056 original/gpt_both3.bin 3224877491 45056 unknown/gpt_both3.bin 3273120389 45056 original/gpt_both4.bin 3273120389 45056 unknown/gpt_both4.bin 3296223989 1007 original/crc32_img_size.txt 3296223989 1007 unknown/crc32_img_size.txt 3346853190 38629376 original/boot-debug-gki.img 3346853190 38629376 unknown/boot-debug-gki.img 3413920804 6586368 original/vendor_boot-debug.img 3413920804 6586368 unknown/vendor_boot-debug.img 3536915896 1705200 original/fastboot_for_mac 3536915896 1705200 unknown/fastboot_for_mac 3718285888 45056 original/gpt_both2.bin 3718285888 45056 unknown/gpt_both2.bin 3762944689 1054 original/crc32_checksum.txt 3762944689 1054 unknown/crc32_checksum.txt 3772440178 14168 original/update_image.bat 3772440178 14168 unknown/update_image.bat 3869862274 10484 original/update_image_for_mac.sh 3869862274 10484 unknown/update_image_for_mac.sh 4046808373 10703200 original/ASUS-xrom.img 4046808373 10703200 unknown/ASUS-xrom.img 4066097203 711720 original/make_f2fs 4066097203 711720 unknown/make_f2fs 4209265207 8192 original/vbmeta.img 4209265207 8192 unknown/vbmeta.img 4294967295 0 original/firmware 4294967295 0 original/lib64 4294967295 0 unknown/firmware 4294967295 0 unknown/lib64 440910305 97792 original/AdbWinApi.dll 440910305 97792 unknown/AdbWinApi.dll 684207671 100663296 original/boot-gki.img 684207671 100663296 unknown/boot-gki.img 785625886 33554432 original/ADF.img 785625886 33554432 unknown/ADF.img 852544587 356 original/multiple_update_image.sh 852544587 356 unknown/multiple_update_image.sh 94533207 313 original/multiple_update_image.bat 94533207 313 unknown/multiple_update_image.bat [ OmniRom 13 Dev - xtrnaw7@t15g ~/test1 ] $ #

[ OmniRom 13 Dev - xtrnaw7@t15g ~/test1 ] $ sort chksum.out | cut -f1 -d " " | uniq -u [ OmniRom 13 Dev - xtrnaw7@t15g ~/test1 ] $

I'm not sure if it's a good idea to install an OS image from an unknown source - I think I would not do that on my phone. But without another option this might be an option

How to analyze the boot loader of an ASUS Zenfone 8

URL: https://xdaforums.com/t/how-to-analyze-the-boot-loader-of-an-asus-zenfone-8.4631111/

How to analyze the boot loader of an ASUS Zenfone 8

Inspired by this blog entry

https://www.pentestpartners.com/security-blog/breaking-the-android-bootloader-on-the-qualcomm-snapdragon-660/

referenced in this XDA thread

https://xdaforums.com/t/cant-unlock-bootloader-failed-to-unlock-your-device-please-try-again-later-24803.4586429/page-2

I have attempted to analyze the bootloader of an ASUS Zenfone 8 using the instructions from the above blog entry. Since the blog entry does not contain details (e.g. exact commands; source URLs for the tools used, etc.), I am documenting my efforts including this information in this forum entry.

It's recommend to read the mentioned blog entry before trying the commands described in this forum entry.
All instructions below are for a PC running the Linux OS.

The boot loader of the ASUS Zenfone 8 is stored in the partition abl_a resp. abl_b

Note:

on my phone the contents of both partitions are equal

1|ASUS_I006D:/ # cksum /sdcard/Download/abl*img3724543824 1048576 /sdcard/Download/abl_a.img 3724543824 1048576 /sdcard/Download/abl_b.img ASUS_I006D:/ #

So the first step is to create an image file of one of the abl_* partitions in an adb shell on the ASUS Zenfone 8 as user root:

dd if=/dev/block/by-name/abl_b of=/sdcard/Download/abl_b.img
and copy the image file to the PC for further investigations:

adb pull sdcard/Download/abl_b.img

If this is not possible (e.g. no root access to the phone) you can use the abl partition from one of the original ASUS firmware images for the ASUS Zenfone 8. The image for the abl partition is part of the file payload.bin in the zip file with the OS image. The image for the abl partition is not part of the custom ROM images for the phone.

Next we check the contents of the image file using the binwalk tool (binwalk is part of the Linux OS - at least in Fedora):

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/asuskey ] $ binwalk abl_b.img DECIMAL       HEXADECIMAL     DESCRIPTION -------------------------------------------------------------------------------- 0             0x0             ELF, 32-bit LSB executable, ARM, version 1 (SYSV) 4512          0x11A0          Certificate in DER format (x509 v3), header length: 4, sequence length: 598 5114          0x13FA          Certificate in DER format (x509 v3), header length: 4, sequence length: 693 5811          0x16B3          Certificate in DER format (x509 v3), header length: 4, sequence length: 730 12288         0x3000          UEFI PI Firmware Volume, volume size: 589824, header size: 0, revision: 0, EFI Firmware File System v2, GUID: 8C8CE578-8A3D-4F1C-3599-896185C32DD312408         0x3078          LZMA compressed data, properties: 0x5D, dictionary size: 16777216 bytes, uncompressed size: 2109640 bytes [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/asuskey ] $

The important part here is the UEFI PI Firmware Volume in the partition. To analyze the UEFI partition the tool uefi-firmware-parser can be used. The uefi-firmware-parser is a Python program that can be downloaded from here.

https://github.com/theopolis/uefi-firmware-parser

To install the parser either download the repository from github or install it direct using this command:
sudo pip install uefi_firmware

Then run the parser for the partition image file:

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/asuskey ] $ uefi-firmware-parser --superbrute abl_b.img Error invalid FV header data (). Error invalid FV header data (). Firmware Volume: 8c8ce578-8a3d-4f1c-9935-896185c32dd3 attr 0x0003feff, rev 2, cksum 0xf047, size 0x90000 (589824 bytes) Firmware Volume Blocks: (1152, 0x200) File 0: 9e21fd93-9c72-4c15-8c4b-e77f1db2d792 type 0x0b, attr 0x00, state 0x07, size 0x47520 (292128 bytes), (firmware volume image)     Section 0: type 0x02, size 0x47508 (292104 bytes) (Guid Defined section)       Guid-Defined: ee4e5898-3914-4259-9d6e-dc7bd79403cf offset= 0x18 attrs= 0x1 (PROCESSING_REQUIRED)         Section 0: type 0x19, size 0x4 (4 bytes) (Raw section)         Section 1: type 0x17, size 0x2030c4 (2109636 bytes) (Firmware volume image section)           Firmware Volume: 8c8ce578-8a3d-4f1c-9935-896185c32dd3 attr 0x0003feff, rev 2, cksum 0x448d, size 0x2030c0 (2109632 bytes)             Firmware Volume Blocks: (32963, 0x40)             File 0: ffffffff-ffff-ffff-ffff-ffffffffffff type 0xf0, attr 0x00, state 0x07, size 0x2c (44 bytes), (ffs padding)             File 1: f536d559-459f-48fa-8bbc-43b554ecae8d type 0x09, attr 0x00, state 0x07, size 0x203038 (2109496 bytes), (application)               Section 0: type 0x15, size 0x1c (28 bytes) (User interface name section)               Name: LinuxLoader               Section 1: type 0x10, size 0x203004 (2109444 bytes) (PE32 image section)RawObject: size= 446464 [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/asuskey ] $

The bootloader is the PE32 image in the partition (PE32 image section).

To extract the PE32 image from the image partition use these commands:
# (optional) create a sub directory for the partition contents # mkdir partdetails cd partdetails # extract all parts of the partition to separate files # uefi-firmware-parser --superbrute -e ../abl_b.img

e.g.

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/asuskey/partdetails ] $ uefi-firmware-parser --superbrute -e ../abl_b.img Error invalid FV header data (). Error invalid FV header data (). Firmware Volume: 8c8ce578-8a3d-4f1c-9935-896185c32dd3 attr 0x0003feff, rev 2, cksum 0xf047, size 0x90000 (589824 bytes) Firmware Volume Blocks: (1152, 0x200) File 0: 9e21fd93-9c72-4c15-8c4b-e77f1db2d792 type 0x0b, attr 0x00, state 0x07, size 0x47520 (292128 bytes), (firmware volume image)     Section 0: type 0x02, size 0x47508 (292104 bytes) (Guid Defined section)       Guid-Defined: ee4e5898-3914-4259-9d6e-dc7bd79403cf offset= 0x18 attrs= 0x1 (PROCESSING_REQUIRED)         Section 0: type 0x19, size 0x4 (4 bytes) (Raw section)         Section 1: type 0x17, size 0x2030c4 (2109636 bytes) (Firmware volume image section)           Firmware Volume: 8c8ce578-8a3d-4f1c-9935-896185c32dd3 attr 0x0003feff, rev 2, cksum 0x448d, size 0x2030c0 (2109632 bytes)             Firmware Volume Blocks: (32963, 0x40)             File 0: ffffffff-ffff-ffff-ffff-ffffffffffff type 0xf0, attr 0x00, state 0x07, size 0x2c (44 bytes), (ffs padding)             File 1: f536d559-459f-48fa-8bbc-43b554ecae8d type 0x09, attr 0x00, state 0x07, size 0x203038 (2109496 bytes), (application)               Section 0: type 0x15, size 0x1c (28 bytes) (User interface name section)               Name: LinuxLoader               Section 1: type 0x10, size 0x203004 (2109444 bytes) (PE32 image section) RawObject: size= 446464 Dumping... Wrote: ./volume-0.fv Wrote: ./volume-0/filesystem.ffs Wrote: ./volume-0/file-9e21fd93-9c72-4c15-8c4b-e77f1db2d792/file.obj Wrote: ./volume-0/file-9e21fd93-9c72-4c15-8c4b-e77f1db2d792/section0.guid Wrote: ./volume-0/file-9e21fd93-9c72-4c15-8c4b-e77f1db2d792/section0/section0.raw Wrote: ./volume-0/file-9e21fd93-9c72-4c15-8c4b-e77f1db2d792/section0/section1.fv Wrote: ./volume-0/file-9e21fd93-9c72-4c15-8c4b-e77f1db2d792/section0/section1/volume-ee4e5898-3914-4259-9d6e-dc7bd79403cf.fv Wrote: ./volume-0/file-9e21fd93-9c72-4c15-8c4b-e77f1db2d792/section0/section1/volume-ee4e5898-3914-4259-9d6e-dc7bd79403cf/filesystem.ffs Wrote: ./volume-0/file-9e21fd93-9c72-4c15-8c4b-e77f1db2d792/section0/section1/volume-ee4e5898-3914-4259-9d6e-dc7bd79403cf/file-ffffffff-ffff-ffff-ffff-ffffffffffff/file.obj Wrote: ./volume-0/file-9e21fd93-9c72-4c15-8c4b-e77f1db2d792/section0/section1/volume-ee4e5898-3914-4259-9d6e-dc7bd79403cf/file-f536d559-459f-48fa-8bbc-43b554ecae8d/file.obj Wrote: ./volume-0/file-9e21fd93-9c72-4c15-8c4b-e77f1db2d792/section0/section1/volume-ee4e5898-3914-4259-9d6e-dc7bd79403cf/file-f536d559-459f-48fa-8bbc-43b554ecae8d/section0.ui Wrote: ./volume-0/file-9e21fd93-9c72-4c15-8c4b-e77f1db2d792/section0/section1/volume-ee4e5898-3914-4259-9d6e-dc7bd79403cf/file-f536d559-459f-48fa-8bbc-43b554ecae8d/section1.peWrote: ./volume-0/file-9e21fd93-9c72-4c15-8c4b-e77f1db2d792/section0/guided.preamble Wrote: ./volume-0/file-9e21fd93-9c72-4c15-8c4b-e77f1db2d792/section0/guided.certs Wrote: ./object-1.raw [ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/asuskey/partdetails ] $

The file with the bootloader code is the file with the extension .pe. For the next step we just copy it to the parent directory:

cp ./volume-0/file-9e21fd93-9c72-4c15-8c4b-e77f1db2d792/section0/section1/volume-ee4e5898-3914-4259-9d6e-dc7bd79403cf/file-f536d559-459f-48fa-8bbc-43b554ecae8d/section1.pe ..

The file section1.pe can then be disassembled using a disassembler supporting ARM CPUs.

Note:

There is a free version of the disassembler ida (https://hex-rays.com) mentioned in the blog entry- but the freeware version of the current ida version does not support ARM CPUs anymore so we can not use it for this task.

One OpenSource disassembler that supports ARM CPUs is radare2.

https://github.com/radareorg/radare2

radare2 can be downloaded from this web page

https://github.com/radareorg/radare2/releases

radare2 is distributed as sourcecode; just use the instructions on the download page to compile it.

Note:

If the installation script for radare2 fails just switch the working directory to the directory with the source code for radare2 and install it using the command

make install

To start radare2 to disassemble the bootloader use this command

r2 section1.pe
e.g

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/asuskey ] $ r2 section1.pe -- THIS IS NOT A BUG [0x00011000]>

The main interface for radare2 is a simple CLI shell; to start a text GUI while in the radare2 CLI use the command:

v<enter>
or start radare2 with these parameter

r2 -c v section1.pe

radare2 also supports a Web GUI -- to start radare2 with the WEB GUI use

r2 -c=H section1.pe

e.g.

[ OmniRom 13 Dev - xtrnaw7@t15g /data/develop/android/asuskey ] $ r2 -c=H section1.pe sh: line 1: http://localhost:9090/m: No such file or directory Starting http server... open http://localhost:9090/ r2 -C http://localhost:9090/cmd/

Use the URL http://localhost:9090/ in a WebBrowser to access the WebGUI from radare2.

Notes:

The WebGUI only works out of the box if radare2 is installed in the default directory /usr/local.

For more infos about how to use radare2 see the online help here: https://book.rada.re/index.html

Binary Ninja is another disassembler supporting ARM CPUs. Binary Ninja contains a free online disassembler in the cloud that can be accessed using the URL https://cloud.binary.ninja/ without any local installations.

I'm not really an expert in disassembling ARM code, so I haven't found any magic function in the bootloader so far, but what I already found is the list of supported fastboot oem commands for the ASUS Zenfone 8:

Supported fastboot oem commands on the ASUS Zenfone 8

oem enable-charger-screen oem disable-charger-screen oem off-mode-charge oem select-display-panel oem device-info oem crc32check_ oem hashsize_ oem partition oem maskid oem cpuid oem asus-lock oem asus-unlock oem asus-rt-unlock oem asus-wm-unlock oem disable_checkfw oem record-info oem gpt-info oem adb_enable oem shutdown oem get-dtid oem get-hwid oem get-prjid oem get-skuid oem get-rfid oem get-featureid oem get-jtagid oem get-bcid oem get-secdispid oem get-cpuidhash oem get-toolid oem isn-info oem ssn-info oem system-info oem get_build_version oem get-batcap oem get-batvol oem get-bootcount oem factory-reset oem factory-reset2 oem reboot-recovery oem set-permissive oem enter-dload oem check-s3 oem EnterShippingMode oem check-nrfuse oem check-fuse oem fuse-info oem show-barcode oem checksetupwizard oem asus-csc_lk oem rsa_test_ oem crc32_ oem hash_ oem gen-random oem auth-hash oem auth-hash_2 oem auth-hash_3 oem get-imeiauth oem slot_b_enable oem get-verify_vbmeta_ret oem update-cmdline_ oem backup-fac oem restore-fac oem get-pmic-reg_ oem write-pmic-reg_ oem reset-boot_count oem reset-lock_count oem reset-a_retry_count oem reset-a_unbootable_count oem reset-b_retry_count oem reset-b_unbootable_count oem force-hwid_ oem reset-dev_info oem reset-auth2 oem reset-auth3 oem disable-verity oem enable-verity oem enable-vbmeta oem read-vbmeta oem read-rollback oem reset-rollback

How to compile the OmniROM 14 for the ASUS Zenfone 8

URL: https://xdaforums.com/t/how-to-compile-omnirom-14-for-the-asus-zenfone-8.4643071/

Update 29.03.2024:

Please see also the Infos for building a CustomROM image for changes in the build process since Q1 2024.

How to compile the OmniROM 14 for the ASUS Zenfone 8

Compiling the CustomROM OmniROM 14 from within Linux is again a very simple and straight forward process -- the developer for the OmniROM did once again a really amazing good job setting up the build environment for the OmniROM 14.

I'm repeating myself here - but it can't be said often enough.

Notes:

This HowTo is about how to compile the OmniROM 14 for the ASUS Zenfone 8 -- see How to compile the OmniROM 13 for the ASUS Zenfone 8 for how to compile the OmniROM 13 for the ASUS Zenfone 8; see How to compile the OmniROM for the ASUS Zenfone 8 for how to compile the OmniROM 12 for the ASUS Zenfone 8.

Hardware requirements

Message from the repo tool:

"The minimum required amount of free memory is around 16GB, and even with that, some configurations may not work."

In my experience, the computer on which OmniROM is compiled should have at least 32 GB RAM and 4 CPU cores.
(The numbers in this document are for a PC with an Intel Core i7 with 16 cores and 128 GB RAM).

Note that the scripts used to create the OmniROM image start one workspace per CPU and each workspace requires about 4 GB of RAM. Plus about 2 GB RAM for the general tasks.

As of 09.01.2024, the repositories for OmniROM 14 occupy about 290 GB of disk space; the additional repositories for the hardware-specific repositories for the ASUS Zenfone 8 occupy about 30 GB of disk space.
The compile process will occupy additional 140 GB for the temporary output files.
In total, the repositories for OmniROM 14 require around 500 GB of storage space (the space used will increase with every "repo sync")

Prerequisites

Some of the repositories for OmniROM 14 are on https://gitlab.com and the access to these repositories for the repository sync is done via ssh. Therefore, to sync these repositories a user account on gitlab.com with configured public ssh key is required. The user account on gitlab.com is free of charge and the public ssh key can be uploaded into the user profile on gitlab.com (see here: https://docs.gitlab.com/ee/user/ssh.html)

The repositories on http://github.com are currently accessed via https but that might change in the future. Therefor it's recommended to create a user on github.com and add the ssh public key to the profile of that user also.

Note:

To check how the access to the repositories is configured use this command

[OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0/.repo ] $ grep fetch $( find . -name "*xml" ) ./local_manifests/local_manifest.xml: fetch="https://github.com/DonkeyCoyote" /> ./manifests/omni-aosp.xml: fetch="https://github.com/" /> ./manifests/omni-aosp.xml: fetch="ssh://git@gitlab.com/omnirom" /> ./manifests/omni-aosp.xml: fetch="https://github.com/omnirom" ./manifests/omni-caf.xml: fetch="https://github.com/" /> ./manifests/omni-caf.xml: fetch="https://github.com/omnirom" ./manifests/omni-caf.xml: fetch="https://git.codelinaro.org/clo/la" ./manifests/omni-private.xml: fetch="ssh://git@github.com/omnirom" ./manifests/default.xml: fetch="https://android.googlesource.com" [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0/.repo ] $

Use the commands

ssh git@github.com ssh git@gitlab.com
to test the access via ssh and add the ssh hostkeys of the hosts github.com and gitab.com to the known_hosts file.

Or use

grep"^gitlab.com"~/.ssh/known_hosts ||ssh-keyscan gitlab.com >>~/.ssh/known_hosts grep"^github.com" ~/.ssh/known_hosts || ssh-keyscan github.com >>~/.ssh/known_hosts
to add the ssh hostkeys of the repository server used to the local known_hosts file of the user used to retrieve the repositories.

It's recommended to start the ssh-agent and load the private ssh keys before starting the sync of the repositories, e.g. if no global ssh-agent is running use:

$( eval ssh-agent ) ssh-add
Use

ssh-add -l

to list the ssh keys already loaded by the ssh-agent.

The tool repo is required to maintain the repositories for the OmniROM 14; you can download the tool "repo" from https://gerrit.googlesource.com/git-repo or download it with this command:

curl https://storage.googleapis.com/git-repo-downloads/repo > ./repo chmod a+x ./repo

It's not necessary to install additional tools for creating an OmniROM 14 image -- all necessary tools to create the image are in the repositories for the OmniROM 14.

The repositories used for the OmniROM 14 use the git command line extension git lfs. At least in Fedora this extension is not installed by default while installing git. In Fedora git lfs can be installed using this command:

sudo dnf install git-lfs

Creating the local copies of the repositories for the OmniROM 14

Most (but not all) of the necessarry steps to sync the repositories necessary to create an OmniROM 14 installation image are described in the section "Getting Started" here: https://github.com/omnirom/android but some instructions are missing the details.

To create a local copy of the repositories for OmniROM 14 these steps are required:

First create a new directory for the repositories and change the working directory to the new directory, e.g.:

mkdir /data/develop/android/OmniROM_14.0 cd /data/develop/android/OmniROM_14.0

Then init the local repositories using the command:

repo init -u https://github.com/omnirom/android.git -b
        android-14.0 --git-lfs

Note that the command repo init only creates a hidden directory called .repo.

Next do the initial sync of the repositories using the command:

repo sync

The initial repository sync will download all repositories necessary to create the OmniROM 14 image except the hardware dependent repositories (these repos are different for the various phone models).

Therefor you must add the repositories for the vendor and device tree for your phone to the manifests in the local repositories after the download finished.

This should be done in the file .repo/local_manifests/local_manifest.xml (just create the directory .repo/local_manifests if it does not exist).

For the ASUS Zenfone 8 the file should look like this:

[ OmniRom 14 Dev - xtrnaw7@t15g
        /data/develop/android/OmniROM_14.0 ] $ cat
          .repo/local_manifests/local_manifest.xml 

        <?xml version="1.0" encoding="UTF-8"?>

        

        <manifest>

        <remote name="DonkeyCoyote"

        fetch="https://github.com/DonkeyCoyote" />

        

        <project path="device/asus/zenfone8"
        name="android_device_asus_zenfone8" remote="omnirom"
        revision="android-14.0" />

        <project path="vendor/asus" name="proprietary_vendor_asus"
        remote="DonkeyCoyote" revision="android-14.0" />

        <project path="vendor/microg"
        name="android_prebuilts_prebuiltapks" remote="omnirom"
        revision="android-13.0" />

        <project path="vendor/images/asus"
        name="android_vendor_images_asus" remote="gitlab"
        revision="android-13.0" />

        

        </manifest>

        [ OmniRom 14 Dev - xtrnaw7@t15g
        /data/develop/android/OmniROM_14.0 ] $

Notes:

The repository android_prebuilts_prebuiltapks is only necessary for OmniROM 14 images that should include the MicroG packages.

After updating the file local_manifest.xml do another

repo sync

to sync the added repositories. This time the download will be faster because only the additional repos will be downloaded.

Now you can start the build with these commands:

# optional:

        #

        # to create an OmniROM image with MicroG packages set the
        environment variable ROM_BUILDTYPE to MICROG before starting the
        build

        #

        # to create an OmniROM image with GAPPS packages set the
        environment variable ROM_BUILDTYPE to GAPPS before starting the
        build

        #(that will only work if the repositories for the GAPPS are
        configured, of course)

        #

        export ROM_BUILDTYPE=MICROG

        

        . build/envsetup.sh

        

        brunch omni_zenfone8-user

in the directory with the downloaded repositories (use the command "breakast" without parameter instead of "brunch" to see other possible targets)

Be aware that the first time the compile process will take 4 hours or more.

If everything is fine there will be a ZIP file with your own Custom OmniROM 14 image in the directory ./out/target/product/zenfone8/ when the compile script is done, e.g. for images with the MicroG packages:

[ OmniRom 14 Dev - xtrnaw7@t15g
        /data/develop/android/OmniROM_14.0 ] $ ls -l
        out/target/product/zenfone8/omni-14-202311*zip

        -rw-rw-r--. 3 xtrnaw7 xtrnaw7 1363298353 Nov 20 10:22
        out/target/product/zenfone8/omni-14-20231119-zenfone8-MICROG.zip

        [ OmniRom 14 Dev - xtrnaw7@t15g
        /data/develop/android/OmniROM_14.0 ] $

The image can now be installed on the ASUS Zenfone 8 using the LineageOS Recovery or TWRP.

Notes

Be aware that each new OmniROM 14 image created with brunch will overwrite the existing OmniROM 14 image (the name of the image is different but the used inodes are the same for all zip files):

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ ls -il out/target/product/zenfone8/omni-14-202311*zip 80813412 -rw-rw-r--. 3 xtrnaw7 xtrnaw7 1363298353 Nov 20 10:22 out/target/product/zenfone8/omni-14-20231119-zenfone8-MICROG.zip 80813412 -rw-rw-r--. 3 xtrnaw7 xtrnaw7 1363298353 Nov 20 10:22 out/target/product/zenfone8/omni-14-20231120-zenfone8-MICROG.zip [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ [OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ cksum out/target/product/zenfone8/omni-14-202311*zip 1217746399 1363298353 out/target/product/zenfone8/omni-14-20231119-zenfone8-MICROG.zip 1217746399 1363298353 out/target/product/zenfone8/omni-14-20231120-zenfone8-MICROG.zip [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

Therefor you should copy the existing zip file with the image to a new directory before building a new image.

Certificates for the OS images

For OS images that should be used on the phone for daily use I strongly recommend to create your own certificates for the OS.

In the OmniROM development directory tree the neccessary files, platform.pk8 and platform.x509.pem, are in the directory

./build/make/target/product/security

in your build tree for the CustomROM, e.g.

[ OmniRom 14 Dev - xtrnaw7@t15g
        /data/develop/android/OmniROM_14.0 ] $ ls -l
        ./build/make/target/product/security/platform.*

        -rw-rw-r--. 1 xtrnaw7 xtrnaw7 1216 Nov 14 15:24
        ./build/make/target/product/security/platform.pk8

        -rw-rw-r--. 1 xtrnaw7 xtrnaw7 1675 Nov 14 15:24
        ./build/make/target/product/security/platform.x509.pem

        [ OmniRom 14 Dev - xtrnaw7@t15g
        /data/develop/android/OmniROM_14.0 ] $

To manually create the keys use these commands:

To manually create the keys use these commands:

Android uses 2048-bit RSA keys with public exponent 3. You can generate certificate/private key pairs using the openssl tool from openssl.org:

# generate RSA key #[xtrnaw7@t15g /data/develop/android/test1]$ openssl genrsa -3 -out temp.pem 2048 [xtrnaw7@t15g /data/develop/android/test1]$ ls -ltr total 4 -rw-------. 1 xtrnaw7 xtrnaw7 1704 Nov 14 13:24 temp.pem # create a certificate with the public part of the key # [xtrnaw7@t15g /data/develop/android/test1]$ openssl req -new -x509 -key temp.pem -out releasekey.x509.pem -days 10000 -subj '/C=DE/ST=Hessen/L=Frankfurt am Main/O=Android/OU=Android/CN=Android/emailAddress=bnsmb01@gmail.com' [xtrnaw7@t15g /data/develop/android/test1]$ [xtrnaw7@t15g /data/develop/android/test1]$ ls -ltr total 8 -rw-------. 1 xtrnaw7 xtrnaw7 1704 Nov 14 13:24 temp.pem -rw-rw-r--. 1 xtrnaw7 xtrnaw7 1456 Nov 14 13:25 releasekey.x509.pem [xtrnaw7@t15g /data/develop/android/test1]$ # create a PKCS#8-formatted version of the private key # [xtrnaw7@t15g /data/develop/android/test1]$ openssl pkcs8 -in temp.pem -topk8 -outform DER -out releasekey.pk8 -nocrypt [xtrnaw7@t15g /data/develop/android/test1]$ [xtrnaw7@t15g /data/develop/android/test1]$ ls -ltr total 12 -rw-------. 1 xtrnaw7 xtrnaw7 1704 Nov 14 13:24 temp.pem -rw-rw-r--. 1 xtrnaw7 xtrnaw7 1456 Nov 14 13:25 releasekey.x509.pem -rw-------. 1 xtrnaw7 xtrnaw7 1218 Nov 14 13:25 releasekey.pk8 [xtrnaw7@t15g /data/develop/android/test1]$ # securely delete the temp.pem file #[xtrnaw7@t15g /data/develop/android/test1]$ shred --remove temp.pem [xtrnaw7@t15g /data/develop/android/test1]$ [xtrnaw7@t15g /data/develop/android/test1]$ ls -ltr total 8 -rw-rw-r--. 1 xtrnaw7 xtrnaw7 1456 Nov 14 13:25 releasekey.x509.pem -rw-------. 1 xtrnaw7 xtrnaw7 1218 Nov 14 13:25 releasekey.pk8 [xtrnaw7@t15g /data/develop/android/test1]$

From the documentation:

The openssl pkcs8 command given above creates a .pk8 file with no password, suitable for use with the build system. To create a .pk8 secured with a password (which you should do for all actual release keys), replace the -nocrypt argument with -passout stdin; then openssl will encrypt the private key with a password read from standard input. No prompt is printed, so if stdin is the terminal the program will appear to hang when it's really just waiting for you to enter a password. Other values can be used for the-passout argument to read the password from other locations; for details, see the openssl documentation.

The temp.pem intermediate file contains the private key without any kind of password protection, so dispose of it thoughtfully when generating release keys. In particular, the GNUshred utility may not be effective on network or journaled filesystems. You can use a working directory located in a RAM disk (such as a tmpfs partition) when generating keys to ensure the intermediates are not inadvertently exposed.

Update 01.07.2024

To view the certificate use this command:

openssl x509 -in ./build/make/target/product/security/platform.x509.pem -text

Updating the local repositories

To update the local repositories change the working directory to the directory with the repositories for OmniROM 14 and execute

repo sync

After the synchronization is complete, create a new operating system image using the commands described above.

It is recommended to create a new image at least once a month after the new security patches for Android are added to the repositories for OmniROM 14.

To view the date of the security patch in the source files for the OmniROM 14 execute this command in the directory with the source files for the OmniROM 14:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ date Tue Jan 9 07:24:05 PM CET 2024 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ grep "^[[:space:]]*PLATFORM_SECURITY_PATCH[[:space:]]*:=" ./build/make/core/version_defaults.mk PLATFORM_SECURITY_PATCH := 2024-01-05 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $
See How to apply the Android Security patches to a local repository for how to apply an Android Security patch to the local repository.

Moving the repositories to another directory

In case you want to move the repositories for OmniROM 14 to another directory or rename the directory with the repositories:

Be aware that the make scripts create some absolute links in the output directory; to list these absolute links use this command while in the directory with the repositories for OmniROM 14 :

find . -type l -ls | grep "$PWD"

e.g. there are currently a lot of absolute links in the output directory:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ find . -type l -ls | grep "$PWD" | wc -l 131245[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

Before starting another build after moving the directories with the repositories you must correct these links (or clean the directory ./out)

Log files

The verbose log of the compile script is always written to a compressed file in the output directory, e.g.:

ls -ltr /data/develop/android/OmniROM_14.0/out/verbose.log*

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ ls -ltr /data/develop/android/OmniROM_14.0/out/verbose.log* -rw-rw-r--. 1 xtrnaw7 xtrnaw7 938 Nov 19 19:00 /data/develop/android/OmniROM_14.0/out/verbose.log.4.gz -rw-rw-r--. 1 xtrnaw7 xtrnaw7 7321648 Nov 19 19:02 /data/develop/android/OmniROM_14.0/out/verbose.log.5.gz -rw-rw-r--. 1 xtrnaw7 xtrnaw7 21660 Nov 19 19:05 /data/develop/android/OmniROM_14.0/out/verbose.log.3.gz -rw-rw-r--. 1 xtrnaw7 xtrnaw7 20241 Nov 19 19:10 /data/develop/android/OmniROM_14.0/out/verbose.log.2.gz -rw-rw-r--. 1 xtrnaw7 xtrnaw7 2917838 Nov 19 20:33 /data/develop/android/OmniROM_14.0/out/verbose.log.1.gz -rw-rw-r--. 1 xtrnaw7 xtrnaw7 73725 Nov 20 10:22 /data/develop/android/OmniROM_14.0/out/verbose.log.gz [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

Other options of the make tools

To create the image with other options use

. build/envsetup.sh lunch $ make <options>

To list the known options for the make command use this command:

make help

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ make help The basic Android build process is: cd /data/develop/android/OmniROM_14.0 source build/envsetup.sh # Add "lunch" (and other utilities and variables) # to the shell environment. lunch [<product>-<variant>] # Choose the device to target. m [<goals>] # Execute the configured build. Usage of "m" imitates usage of the program "make". See /data/develop/android/OmniROM_14.0/build/make/Usage.txt for more info about build usage and concepts. The parallelism of the build can be set with a -jN argument to "m". If you don't provide a -j argument, the build system automatically selects a parallel task count that it thinks is optimal for your system. Common goals are: clean (aka clobber) equivalent to rm -rf out/ checkbuild Build every module defined in the source tree droid Default target sync Build everything in the default target except the images, for use with adb sync. nothing Do not build anything, just parse and validate the build structure java Build all the java code in the source tree native Build all the native code in the source tree host Build all the host code (not to be run on a device) in the source tree target Build all the target code (to be run on the device) in the source tree (java|native)-(host|target) (host|target)-(java|native) Build the intersection of the two given arguments snod Quickly rebuild the system image from built packages Stands for "System, NO Dependencies" vnod Quickly rebuild the vendor image from built packages Stands for "Vendor, NO Dependencies" pnod Quickly rebuild the product image from built packages Stands for "Product, NO Dependencies" senod Quickly rebuild the system_ext image from built packages Stands for "SystemExt, NO Dependencies" onod Quickly rebuild the odm image from built packages Stands for "Odm, NO Dependencies" vdnod Quickly rebuild the vendor_dlkm image from built packages Stands for "VendorDlkm, NO Dependencies" odnod Quickly rebuild the odm_dlkm image from built packages Stands for "OdmDlkm, NO Dependencies" sdnod Quickly rebuild the system_dlkm image from built packages Stands for "SystemDlkm, NO Dependencies" So, for example, you could run: cd /data/develop/android/OmniROM_14.0 source build/envsetup.sh lunch aosp_arm-userdebug m -j java to build all of the java code for the userdebug variant of the aosp_arm device. #### build completed successfully (4 seconds) #### [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

Creating only the kernel or the boot image

To create only the kernel or the boot image use these commands:

. build/envsetup

# select the target hardware

#

breakfast omni_zenfone8-user

 # Now instead of following the rest of the guide
        (which will compile both Android and the kernel

# for your device), simply compile the kernel:

#

mka kernel

 # This command will also generate the device tree,
        initramfs and boot.img images

#

mka bootimage

Trouble Shooting

see Some hints for repo sync troubleshooting for how to fix some common errors while executing the command "repo sync".

see Some hints for fixing errros building an Custom ROM Android image for how to fix some common errors while creating the image

TWRP and OmniROM 14

Update 10.01.2024

For OmniROM 14 and other Android 14 based CustomROMs on the ASUS Zenfone 8 the TWRP version 3.7.0.12-1 or newer is required (because TWRP version 3.7.0.12-0 or older can not mount the /data partitions created by Android 14). TWRP version 3.7.0.12-1 can now be downloaded from the TWRP webpage:

https://eu.dl.twrp.me/I006D/

Notes:

After dirty flashing an installed OmniROM 13 with OmniROM 14 the current official TWRP 3.7.0.12-0 can still access the /data partition - the "problem" only exists after a clean installation of OmniROM 14 (= with factory reset).

Update 10.01.2024

This workaround is not necessary anymore with TWRP version 3.7.0.12-1 or newer

Installing OmniROM 14 via script using the official TWRP image 3.7.0.12-0 for the ASUS Zenfone 8

The installation and customization of OmniROM 14 via script, as described in here How to install and configure the Android OS via Script, using the TWRP image for the ASUS Zenfone 8 3.7.0_12-0 will only partially work due to the missing support for Android 14 in that release of TWRP.

A workaround to install OmniROM 14 using the TWRP image without OmniROM 14 support is to use the scripts to install and customize OmniROM 13 first, and then use the scripts to update to OmniROM 14 , e.g.:
#install and customize OmniROM 13 # ./prepare_phone.sh in all # update to OmniROM 14 # # the name of the image must be a fully qualified path # export OMNIROM_14_OS_IMAGE_TO_INSTALL="/data/backup/ASUS_ZENFONE8/omnirom_local/omni-14/omni-14-20231119-zenfone8-MICROG.zip" ./prepare_phone.sh install_os:noreboot:${OMNIROM_14_OS_IMAGE_TO_INSTALL} install_twrp:noreboot:next install_magisk_v26_in_boot_partition:reboot:next
Note: install_magisk_via_twrp.sh v3.1.3.1 or newer is required for this approach

History of this entry

09.01.2024

the repositories for OmniROM 14 are now final and can be used to create an OmniROM 14 installation image for the ASUS Zenfone 8

Updated the section about TWRP for OmniROM 14

10.01.2024

The section about TWRP for OmniROM 14 was rewritten;, there is now an official TWRP version for the ASUS Zenfone 8 with support for Android 14.

21.01.2024

Added note on handling 429 errors of the repo sync command

22.01.2024

Added info about the mandatory git command line extension "git lfs"

08.02.2024

moved the trouble shooting instructions to separate posts

31.03.2024

added a reference to a new post about changes in the build process in Q1 2024

01.07.2024

added the command to view the contents of a .pem file

How to get or set the lock screen settings via CLI command

URL: https://xdaforums.com/t/how-to-get-or-set-the-lock-screen-settings-via-cli-command.4647038/

How to get or set the lock screen settings via CLI command

The Android CLI command

cmd lock_settings

can be used to get or set the lock screen settings in an adb session or a script. The command should be executed by the default user in the adb session (-> no root access necessary).

The known options for the command cmd lock_settings are

cmd lock_settings help

ASUS_I006D:/ $ cmd lock_settings helplockSettings service commands: NOTE: when lock screen is set, all commands require the --old <CREDENTIAL> argument. help Prints this help text. get-disabled [--old <CREDENTIAL>] [--user USER_ID] Checks whether lock screen is disabled. set-disabled [--old <CREDENTIAL>] [--user USER_ID] <true|false> When true, disables lock screen. set-pattern [--old <CREDENTIAL>] [--user USER_ID] <PATTERN> Sets the lock screen as pattern, using the given PATTERN to unlock. set-pin [--old <CREDENTIAL>] [--user USER_ID] <PIN> Sets the lock screen as PIN, using the given PIN to unlock. set-password [--old <CREDENTIAL>] [--user USER_ID] <PASSWORD> Sets the lock screen as password, using the given PASSOWRD to unlock. clear [--old <CREDENTIAL>] [--user USER_ID] Clears the lock credentials. verify [--old <CREDENTIAL>] [--user USER_ID] Verifies the lock credentials. remove-cache [--user USER_ID] Removes cached unified challenge for the managed profile. set-resume-on-reboot-provider-package <package_name> Sets the package name for server based resume on reboot service provider. require-strong-auth [--user USER_ID] <reason> Requires the strong authentication. The current supported reasons: STRONG_AUTH_REQUIRED_AFTER_USER_LOCKDOWN. ASUS_I006D:/ $

To define a pin for the lock screen use

cmd lock_settings set-pin <new_pin>

e.g. to set the pin to "12345" use:
ASUS_I006D:/ $ cmd lock_settings set-pin 12345 Pin set to '12345' ASUS_I006D:/ $
To define a password for the lock screen use

cmd lock_settings set-password <new_password>

e.g. to set the password to "start" use:

ASUS_I006D:/ $ cmd lock_settings set-password start Password set to 'start' ASUS_I006D:/ $

To change an existing pin or password the additional parameter --old <old_credentials> is required

E.g. to change the pin from "12345" to "45678" use

ASUS_I006D:/ $ cmd lock_settings set-pin --old 12345 45678 Pin set to '45678' ASUS_I006D:/ $

To verify the lock screen settings use

cmd lock settings verify --old <pin_or_password>

e.g . if the current pin is 45678 :

ASUS_I006D:/ $ cmd lock_settings verify --old 45678 Lock credential verified successfully ASUS_I006D:/ $ cmd lock_settings verify --old 12345 Old password '12345' didn't match 255|ASUS_I006D:/ $

To clear the lock settings use

cmd lock_settings --old <pin_or_password> clear

Note: The old credentials are mandatory for this command

e.g

ASUS_I006D:/ $ cmd lock_settings clear --old 45678 Lock credential cleared ASUS_I006D:/ $ ASUS_I006D:/ $ cmd lock_settings verify --old 45678 Old password provided but user has no password 255|ASUS_I006D:/ $

To check if a screen lock is defined use

cmd lock_settings verify --old <any_password_or_pin>

e.g:

ASUS_I006D:/ $ cmd lock_settings verify --old 45678 ; echo $?Old password provided but user has no password 255 ASUS_I006D:/

If the provided credentials match the configured lock settings the output is

ASUS_I006D:/ $ cmd lock_settings verify --old 45678 ; echo $?Lock credential verified successfully 0 ASUS_I006D:/ $

If the there is a pin or password set but the credentials used in the command are wrong the output of the command is:

ASUS_I006D:/ $ cmd lock_settings verify --old 45679 ; echo $?Old password '45679' didn't match 255 ASUS_I006D:/ $

If you have entered incorrect old login data more than 5 times, the command issues the message Request throttled:

ASUS_I006D:/ $ cmd lock_settings set-pin --old 12376 12345Request throttled 255|ASUS_I006D:/ $

and you must wait 30 seconds (or more - this value is increased dynamically - see below) before you can start another attempt

Note:

The command cmd lock_settings can also be used to define a screen pattern for the lock screen using the parameter set-pattern.

To enable or disable the locked screen on the phone in an adb session use the command input.

To lock the screen in an adb session via a CLI command use :

input keyevent 26

To wake up the phone use:

input keyevent 82

To unlock the phone screen via a CLI command use (assuming the pin used is "12345"):

input keyevent 82 ; input swipe 930 880 930 380 10 ; input text 12345 ; input keyevent 66

Note:

The parameter for the command "input swipe" in the example work for the ASUS Zenfone 8 - other values might be required for other phones.

To unlock the phone screen if no password, pin, or pattern is defined use:

input keyevent 82 && input keyevent 66

Note:

Credits for the input command examples go to: https://www.pentestpartners.com/blog/brute-forcing-android-pins-or-why-you-should-never-enable-adb-part-1/

The usage help for the command input is

input help

ASUS_I006D:/ $ input help Usage: input [<source>] [-d DISPLAY_ID] <command> [<arg>...] The sources are: touchnavigation touchscreen joystick stylus touchpad gamepad dpad mouse keyboard trackball -d: specify the display ID. (Default: -1 for key event, 0 for motion event if not specified.) The commands and default sources are: text <string> (Default: touchscreen) keyevent [--longpress|--doubletap] <key code number or name> ... (Default: keyboard) tap <x> <y> (Default: touchscreen) swipe <x1> <y1> <x2> <y2> [duration(ms)] (Default: touchscreen) draganddrop <x1> <y1> <x2> <y2> [duration(ms)] (Default: touchscreen) press (Default: trackball) roll <dx> <dy> (Default: trackball) motionevent <DOWN|UP|MOVE|CANCEL> <x> <y> (Default: touchscreen) keycombination [-t duration(ms)] <key code 1> <key code 2> ... (Default: keyboard, the key order is important here.) ASUS_I006D:/ $

A list of known values for keyevent is here http://www.temblast.com/ref/akeyscode.htm or in the Android documentation https://developer.android.com/reference/android/view/KeyEvent

As far as I know there is no parameter for the command cmd lock_settings to clear or read unknown credentials.

In Android 14 (and previous Android versions with file based encryption) removing the credentials to unlock the phone by deleting one or more files no longer works.

The commands mentioned in various blogs on the internet to clear the credentials :

su - -c "mv /data/system/locksettings.db /data/system/locksettings.db.old" ; reboot

or

su - -c "rm /data /system/*.key"

do not work anymore in Android versions using file based encryption.

And even if it would work, it does not help because the data on the phone is encrypted with these credentials.

Brute forcing the password using the command "cmd lock_settings" in an adb shell is in principle possible but because the necessary time between the unsuccessfull attempts will increase exponential a brute force attack against a 4 digit PIN would take around 27 years to complete in the worst case.

This is also true for using the command "trwp decrypt" while booted into the TWRP recovery.

For those interested in the details:

A detailed description about the implementation of the file based encryption in Android can be found here:

https://security.stackexchange.com/questions/196230/connection-between-pin-password-and-encryption-keys-in-android

The only method I currently know of to get rid of an unknown PIN/password is to reset the phone to factory settings. However, this will delete all data on the phone. So it is only an option for phones without important data or if you do have backup of the data on the phone.

Test Environment

OmniROM 14 (= Android 14)
OmniROM 13 (= Android 13)
OmniROM 12 (= Android 12)

History of this entry

25.12.2023

The details about the throttling were wrong - fixed

26.12.2023

Added more details about why it is not possible to remove unknown credentials without data loss

How to enable and configure the Magisk DenyList via CLI commands

URL: https://xdaforums.com/t/how-to-enable-and-configure-the-magisk-denylist-via-cli-commands.4648761/

How to enable and configure the Magisk DenyList via CLI commands

Update 29.01.2024

The DenyList can also be changed using the parameter --denylist of the magisk executable:

magisk --denylist

ASUS_I006D:/ # magisk --denylist DenyList Config CLI Usage: magisk --denylist [action [arguments...] ] Actions: status Return the enforcement status enable Enable denylist enforcement disable Disable denylist enforcement add PKG [PROC] Add a new target to the denylist rm PKG [PROC] Remove target(s) from the denylist ls Print the current denylist exec CMDs... Execute commands in isolated mount namespace and do all unmounts 1|ASUS_I006D:/ #

Magisk saves the settings for the DenyList in the database /data/adb/magisk.db. This is an SQLite database that can also be changed in a script with the executable magisk; the use of magisk for this purpose is:

magisk --sqlite SQL

where "SQL" is the SQL statement to execute. For example to list all tables in the Magisk database /data/adb/magisk.db use:

magisk --sqlite ' SELECT * FROM sqlite_master where type="table";'

e.g.

ASUS_I006D:/ # magisk --sqlite ' SELECT * FROM sqlite_master where type="table";' name=policies|rootpage=3|sql=CREATE TABLE policies (uid INT, policy INT, until INT, logging INT, notification INT, PRIMARY KEY(uid))|tbl_name=policies|type=table name=settings|rootpage=5|sql=CREATE TABLE settings (key TEXT, value INT, PRIMARY KEY(key))|tbl_name=settings|type=table name=strings|rootpage=7|sql=CREATE TABLE strings (key TEXT, value TEXT, PRIMARY KEY(key))|tbl_name=strings|type=table name=denylist|rootpage=9|sql=CREATE TABLE denylist (package_name TEXT, process TEXT, PRIMARY KEY(package_name, process))|tbl_name=denylist|type=table ASUS_I006D:/ # # or using the executable sqlite3 (see below) 1|ASUS_I006D:/ # sqlite3 -table /data/adb/magisk.db ' SELECT * FROM sqlite_master where type="table";' +-------+----------+----------+----------+---------------------------------------------------------------------------------------------------------+ | type | name | tbl_name | rootpage | sql | +-------+----------+----------+----------+---------------------------------------------------------------------------------------------------------+ | table | policies | policies | 3 | CREATE TABLE policies (uid INT, policy INT, until INT, logging INT, notification INT, PRIMARY KEY(uid)) | | table | settings | settings | 5 | CREATE TABLE settings (key TEXT, value INT, PRIMARY KEY(key)) | | table | strings | strings | 7 | CREATE TABLE strings (key TEXT, value TEXT, PRIMARY KEY(key)) | | table | denylist | denylist | 9 | CREATE TABLE denylist (package_name TEXT, process TEXT, PRIMARY KEY(package_name, process)) | +-------+----------+----------+----------+---------------------------------------------------------------------------------------------------------+ ASUS_I006D:/ #

Note:

All commands to read or change the file /data/adb/magisk.db must be executed as user root.

The SQL database used by the magisk executable is fixed and can not be changed.

See the Magisk documentation for more infos about the DenyList - this HowTo is only about how to configure the DenyList via CLI commands.

Prerequisites

In the current version of Magisk (v26.x) the feature zygisk must be enabled to use the DenyList.

To enable the zygisk feature from Magisk use:

magisk --sqlite 'insert into settings (key,value) values("zygisk","1");'

and reboot the phone.

Notes:

If the database entry for zygisk already exists with another value either delete and recreate it or use

magisk --sqlite 'replace into settings (key,value) values("zygisk","1");'

to change the value.

Use

magisk --sqlite 'select value from settings where (key="zygisk"); '

to read the current value for the setting, e.g.

ASUS_I006D:/ # magisk --sqlite 'select value from settings where (key="zygisk"); ' value=1 ASUS_I006D:/ #
Either delete the entry or change the value to "0" to disable the zygisk feature again.

A reboot between enabling zygisk and enabling the DenyList is not necessary.

Enable and configure the DenyList

To enable the DenyList from Magisk use:

magisk --sqlite 'insert into settings (key,value) values("denylist","1");'

and reboot the phone.

Notes:

If the database entry for the DenyList already exists with another value either delete and recreate it or change the value to "1" (see the description about the "zygisk" entry above for how to do that).

Disabling the DenyList will not delete the list of already configured entries in the DenyList (but they will be ignored if the DenyList is disabled).

To list the current entries in the DenyList use

magisk --sqlite 'select * from denylist'

e.g.

ASUS_I006D:/ # magisk --sqlite 'select * from denylist' package_name=io.github.vvb2060.keyattestation|process=io.github.vvb2060.keyattestation ASUS_I006D:/ #

To add an entry to the DenyList use

magisk --sqlite 'insert into denylist (package_name,process) values("com.mixplorer","com.mixplorer");'

and check the result:

ASUS_I006D:/ # magisk --sqlite 'select * from denylist ; package_name=io.github.vvb2060.keyattestation|process=io.github.vvb2060.keyattestation package_name=com.matoski.adbm|process=com.matoski.adbm package_name=com.apk.editor|process=com.apk.editor package_name=com.mixplorer|process=com.mixplorer ASUS_I006D:/ #

To delete an entry from the DenyList use

magisk --sqlite 'delete from denylist where (package_name="com.apk.editor");'

To find the correct values for the entries in the DenyList use the command

pm list packages

e.g.:

ASUS_I006D:/ # pm list packages -f | grep gallery package:/product/app/Gallery2/Gallery2.apk=com.android.gallery3d package:/system/priv-app/AsusGallery/AsusGallery.apk=com.asus.gallery ASUS_I006D:/ #

Or check the list of applications that can be added to the DenyList in the Magisk App:

Note that you can also add entries for not installed apps into the DenyList.

A reboot is necessary after all changes for the table denylist are done to activate the changes (there might be a method to force Magisk to re-read the database without a reboot but I do not know how this can be done)

Notes:

All changes to the sql database for Magisk can also be done use the sqlite3 executable - e.g.:

# list the tables in the database # sqlite3 -table /data/adb/magisk.db ".tables" # or sqlite3 -table /data/adb/magisk.db ' SELECT * FROM sqlite_master where type="table";' # list the entries in the DenyList # sqlite3 -table /data/adb/magisk.db "select * from denylist ;" # add an entry to the DenyList # sqlite3 -table /data/adb/magisk.db 'INSERT INTO denylist (package_name,process) values("com.apk.editor","com.apk.editor");' # delete an etry from the DenyList # sqlite3 -table /data/adb/magisk.db 'DELETE FROM denylist where (package_name="com.apk.editor");'

A sqlite3 binary for arm64 CPUs is available on my home page: https://bnsmb.de/files/public/Android/sqlite3

see Some hints for using Magisk on Android phones and How to enable root access using Magisk in a script for other usage hints for the Magisk database

How to apply the Android Security patches to a local repository

URL: https://xdaforums.com/t/how-to-apply-the-android-security-patches-to-a-local-repository.4651526/

How to apply the Android Security Patches to a local repository

If you are using a local repository for an Android-based operating system to create your own image of a custom ROM, it is important to add the Android Security Patches released by Google on a monthly basis. For custom ROMs that are actively maintained, this is done by the maintainer of the custom ROM and you can simply get them via an "repo sync" after the patches have been applied by the maintainer.

However, if the custom ROM is no longer actively supported, you will need to do this yourself (as long as Google releases security patches for the Android version used by the custom ROM).

Unfortunately, there are no detailed instructions on how to apply the Android Security Patch to a local repository (... at least aunt Google didn't find a good description)

This entry describes how to apply an Android Security Patch Level to the local repositories for the OmniROM.

(see also Observations while checking how to apply the Android Security patches to a local repository)

To apply an Android Security Patch Level to the local OmniROM repositories the script

https://github.com/omnirom/android_vendor_omni/blob/android-14.0/utils/aosp-merge.sh

can be used. This script applies all changes to OmniROM repositories that are only forked from the original AOSP repositories.

The list of forked AOSP repositories for the OmniROM used by the script aosp-merge.sh is available here:

https://github.com/omnirom/android_vendor_omni/blob/android-14.0/utils/aosp-forked-list

The script and the list are already part of the OmniROM repositories so no need to download them:

xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ ls -l vendor/omni/utils/ total 56 -rw-r--r--. 1 xtrnaw7 xtrnaw7 1159 Nov 20 19:04 Android.mk-rw-r--r--. 1 xtrnaw7 xtrnaw7 480 Nov 20 19:04 aosp-forked-list-rwxr-xr-x. 1 xtrnaw7 xtrnaw7 2528 Nov 20 19:04 aosp-merge.sh-rwxr-xr-x. 1 xtrnaw7 xtrnaw7 1927 Dec 12 15:16 aosp-push-merge.sh -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 2532 Nov 20 19:04 caf-merge.sh -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 1847 Nov 20 19:04 caf-push-merge.sh -rw-r--r--. 1 xtrnaw7 xtrnaw7 135 Nov 20 19:04 caf-repos-list -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 1942 Nov 20 19:04 fork_aosp.sh -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 1966 Nov 20 19:04 fork_github.sh -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 8350 Nov 20 19:04 kernel-upstream.sh drwxr-xr-x. 5 xtrnaw7 xtrnaw7 4096 Nov 20 19:04 omni_emulator -rw-r--r--. 1 xtrnaw7 xtrnaw7 1129 Nov 20 19:04 README.md xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$

To use the script to apply an Android Security Patch Level do:
# create a backup of the existing default.xml (optional) # cp .repo/manifests/default.xml .repo/manifests/default.xml.org.$( date +%Y-%m-%d) # correct the tag in the file default.xml; in this example android-14.0.0_r27 is the tag for the new security patch
# sed -i -e "s/android-14.0.0_r[0-9][0-9]/android-14.0.0_r27/g" .repo/manifests/default.xml # check the result: # diff .repo/manifests/default.xml .repo/manifests/default.xml.org.$( date +%Y-%m-%d) # start the merge of the security patch # cd <top_of_omnirom_repository_directories> cd vendor/omni/utils ./aosp-merge.sh

and enter the git tag for the Android Security Patch level that should be applied (e.g. android-14.0.0_r27 for the security patch 2024-02-05 for Android 14).

Answer the question "Do you want to merge it like squash ? type yes or no" with yes.

See here for an example log output from the script.

If the scripts logs some merge conflicts, you must resolve them manually -- see the file ./vendor/omni/utils/README.md for how to handle merge conflicts.

When done check the result - the value for PLATFORM_SECURITY_PATCH should now contain the new patch level:

xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ grep '^[[:space:]]*PLATFORM_SECURITY_PATCH[[:space:]]*:=' ./build/make/core/version_defaults.mk     PLATFORM_SECURITY_PATCH := 2024-02-05 xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$

Update 04.04.2024 :

Since QPR2 for Android 14 released in Q1 2024 the security patch level is not defined in the file ./build/make/core/version_defaults.mk anymore. Therefor the grep command used above to print the security patch level fails.

To print the security patch level configured in a local Android repository tree the script scripts_on_linux/print_security_patch can be used:

Example:

[ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_14.0 ] $   ../scripts_on_linux/print_security_patch The repository uses the new definitions The current security patch is "2024-04-05" The current build_id is "ap1a" [ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_14.0 ] $

(see Infos for building a CustomROM image and/or check the script source code for details)

If the security patch level is not updated after applying the new security patch after QPR2 was applied, check that the repository build/release is listed in the file ./vendor/omni/utils/aosp-forked-list.

Notes:

The list of available Android Security Patches and the git tags used for them is available on this Website:

https://source.android.com/docs/setup/reference/build-numbers#source-code-tags-and-builds

and fyi:

The Android Security Bulletins are listed here

https://source.android.com/docs/security/bulletin

The source file for the file versions_defaults.mk used to configure the Android Security Patch level before applying QPR2 for a branch is

https://android.googlesource.com/platform/build/+/refs/tags/<branch>/core/version_defaults.mk

e.g. for the branch android-14.0.0_r27:

https://android.googlesource.com/platform/build/+/refs/tags/android-14.0.0_r27/core/version_defaults.mk

If the merge does not work as expected, add some debug code (the echo commands in the excerpt below) to the script aosp-merge.sh:

xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0/vendor/omni/utils$ grep -B2 -A4 "++++" aosp-merge.sh     ret=$(git pull https://android.googlesource.com/platform/$aosp_project ${ref} 2>&1);     echo "++++"     echo "$ret"     echo "----"     if echo $ret | grep "CONFLICT (content)" > /dev/null ; then xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0/vendor/omni/utils$

The configuration of the Android Security Patch level after applying the QPR2 is a little more complicated -- see the source code of the script scripts_on_linux/print_security_patch for details.

The git option pull.rebase must be false for updating the AOSP repositories. Use

git config -l

to check the current value, e.g:
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ git config -l filter.lfs.clean=git-lfs clean -- %f filter.lfs.smudge=git-lfs smudge -- %f filter.lfs.process=git-lfs filter-process filter.lfs.required=true color.ui=auto filter.lfs.clean=git-lfs clean -- %f filter.lfs.smudge=git-lfs smudge -- %f filter.lfs.process=git-lfs filter-process filter.lfs.required=true pull.rebase=false [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

Use

git config --global pull.rebase false

to correct the option (The option can also be set for each repository if the global setting is not feasible)

see How to apply an Android patchlevel to the repositories for the StatiXOS for instructions to apply a patchset to the repositories for the StatixOS.

Update 14.07.2024

In the StatixOS the script to apply an Android Security Patch level is

./vendor/statix/scripts/merge-aosp.py

Example:

The usage to apply the patchlevel android-14.0.0_r53 is:

cd ./vendor/statix/scripts

python ./merge-aosp.py 14.0.0_r53

Observations while checking how to apply the Android Security patches to a local repository

URL: not yet published

Observations while checking how to apply the Android Securtiy patches to a local repository

In this section I describe what I discovered while trying to figure out how to apply an Andord Security patch level to a local Android repository tree.

Note that the following descriptions are for the OmniROM, but I assume it should be more or less the same for other Android-based Custom ROMs.

All custom ROMs based on Android are based on the AOSP repositories published by Google. The AOSP repositories are available at these URLs:

https://android.googlesource.com

and

https://android.googlesource.com/platform

Google uses git tags in the AOSP repositories to mark the Android Security Patches. These git tags use the format

android-<android_version>_r<number>

The list of available Android Security Patches and the git tags used for them is available on this Website:

https://source.android.com/docs/setup/reference/build-numbers#source-code-tags-and-builds

(see the column Tag in the table "Source code tags and builds" on that page)

So for example, the git tag for the Android Security Patch 2024-01-05 for Android 14 is android-14.0.0_r21.

As of 17.01.2024 the list of available git tags for Android Security Patches for Android 14 are:

git tag -l "android-14*"

xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0/manifest$ git tag -l "android-14*" android-14.0.0_r1 android-14.0.0_r10 android-14.0.0_r11 android-14.0.0_r12 android-14.0.0_r13 android-14.0.0_r14 android-14.0.0_r15 android-14.0.0_r16 android-14.0.0_r17 android-14.0.0_r18 android-14.0.0_r19 android-14.0.0_r2 android-14.0.0_r20 android-14.0.0_r21 android-14.0.0_r3 android-14.0.0_r4 android-14.0.0_r5 android-14.0.0_r6 android-14.0.0_r7 android-14.0.0_r8 android-14.0.0_r9 xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0/manifest$ Note: The repository "manifest" is created by my script apply a security patch -- see below (that repo is NOT part of the original repos for OmniROM)

The Android Security Patch of a local repository for the OmniROM can be find in these files:

The current Android Security Patch is stored in the file ./build/make/core/version_defaults.mk:

xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ grep "^[[:space:]]*PLATFORM_SECURITY_PATCH[[:space:]]*:=" ./build/make/core/version_defaults.mk PLATFORM_SECURITY_PATCH := 2023-12-05 xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$

The build id is defined in the file ./build/make/core/build_id.mk:

xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ grep ^BUILD_ID build/make/core/build_id.mk BUILD_ID=UQ1A.240105.004.A1xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$
Update 04.04.2024 : see here for changes in the config files after applying QPR2.

And the git tag used for the local repository is defined in the file .repo/manifests/default.xml:
xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ grep revision= ./.repo/manifests/default.xml <default revision="refs/tags/android-14.0.0_r21" <superproject name="platform/superproject" remote="aosp" revision="android-14.0.0_r21"/> xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$

The list of repositories used in OmniROM cloned from the original AOSP repositories are configured in the manifest file ./.repo/manifests/omni-aosp.xml.

To apply an Android Security Patch to a local repository, the files for this Android Security Patch must be retrieved from the original ASOP repositories and merged into the local repositories for all repositories that were cloned from the original AOSP repositories.

In principle, this can be done with this approach for the OmnROM:

for each repository configued in the manifest file omni-aosp.xml
        do 

          is the repository in the list of forked AOSP repositories
        for the OmniROM?  If not, ignore it 

        

          is the repository in the list of repositories to ignore ?
        if yes, ignore it 

        

          get the URL of the original AOSP repository for this
        repository 

          

          either 

            fetch the changes in the files for the new
        security patch using "git fetch <AOSP_repository_url>
        <new_tag>" 

            merge the files just fetched using "git merge
        <new_tag>" 

          or 

            pull the repository from the AOSP
        repositories using "git pull <AOSP_repository_url>
        <new_tag>"

done

Note:

The list of forked AOSP repositories for the OmniROM is avaiable here:

https://github.com/omnirom/android_vendor_omni/blob/android-14.0/utils/aosp-forked-list

Or in the local OmniROM repository in the file vendor/omni/utils/aosp-forked-list.

A useful command to execute the git command for all related repositories is

for i in $( grep "remote=\"omnirom\"" .repo/manifests/omni-aosp.xml | awk '{print $2}' | awk -F '"' '{print $2}' ) ; do echo "*** $i:" ; ( cd $i && git <git_parameter> ); done

git example

xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ for i in $( grep "remote=\"omnirom\"" .repo/manifests/omni-aosp.xml | awk '{print $2}' | awk -F '"' '{print $2}' ) ; do echo "*** $i:" ; ( cd $i && git status ); done *** build/make: HEAD detached from 8618dc57f5 nothing to commit, working tree clean *** build/soong: HEAD detached from d0db93444 nothing to commit, working tree clean *** bionic: HEAD detached at 8a48794d6 nothing to commit, working tree clean *** device/google/gs101: HEAD detached at 4702794 nothing to commit, working tree clean *** device/google/gs101-sepolicy: HEAD detached at b816d25 nothing to commit, working tree clean *** device/google/gs201: Not currently on any branch. nothing to commit, working tree clean *** device/google/gs201-sepolicy: Not currently on any branch. nothing to commit, working tree clean *** device/google/gs-common: HEAD detached at 5b780d0 nothing to commit, working tree clean *** device/google/pantah: Not currently on any branch. nothing to commit, working tree clean *** device/google/raviole: HEAD detached at fb0bf33 nothing to commit, working tree clean *** external/drm_hwcomposer: Not currently on any branch. nothing to commit, working tree clean *** external/libdrm: HEAD detached at b37494c0 nothing to commit, working tree clean *** external/wpa_supplicant_8: HEAD detached at 5ec10e02 nothing to commit, working tree clean *** frameworks/av: HEAD detached from 3438afad77 nothing to commit, working tree clean *** frameworks/base: HEAD detached from bbff1be6f9f6 nothing to commit, working tree clean *** frameworks/native: HEAD detached from c065ae34cf nothing to commit, working tree clean *** frameworks/opt/telephony: HEAD detached at 9dc4b919c1 nothing to commit, working tree clean *** hardware/interfaces: HEAD detached at 092e15a1a nothing to commit, working tree clean *** hardware/libhardware: HEAD detached at b74a1c48 nothing to commit, working tree clean *** hardware/google/pixel-sepolicy: Not currently on any branch. nothing to commit, working tree clean *** packages/apps/Contacts: HEAD detached at e408d4996 nothing to commit, working tree clean *** packages/apps/Dialer: Not currently on any branch. nothing to commit, working tree clean *** packages/apps/Launcher3: HEAD detached from c78e666745 nothing to commit, working tree clean *** packages/apps/Messaging: HEAD detached from d10e7f2 nothing to commit, working tree clean *** packages/apps/Settings: HEAD detached from 2b08c3c1dd nothing to commit, working tree clean *** packages/apps/SettingsIntelligence: HEAD detached at f699f93 nothing to commit, working tree clean *** packages/inputmethods/LatinIME: Not currently on any branch. nothing to commit, working tree clean *** packages/services/Telecomm: HEAD detached at f736871fc nothing to commit, working tree clean *** packages/services/Telephony: HEAD detached at 16a503c5a nothing to commit, working tree clean *** system/core: HEAD detached at 50429dbb81 nothing to commit, working tree clean *** system/libhidl: HEAD detached at 5b0b596 nothing to commit, working tree clean *** system/security: HEAD detached at 0d99cffc nothing to commit, working tree clean *** system/sepolicy: HEAD detached from dece3aa34 nothing to commit, working tree clean *** system/update_engine: HEAD detached at 766cb7ff nothing to commit, working tree clean *** system/vold: HEAD detached from ec85857 nothing to commit, working tree clean *** system/timezone: HEAD detached at f344032b nothing to commit, working tree clean *** packages/modules/Bluetooth: HEAD detached at c3717d7581 nothing to commit, working tree clean *** packages/modules/Wifi: HEAD detached at 9062be6fd1 nothing to commit, working tree clean xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$

Most changed files from the AOSP repositories can be automatically merged into the local repository using the git command, but conflicts may occur with some files. These conflicts must be resolved manually in order to merge the tag.

For files with CONFLICTS edit the file (search for HEAD), remove the conflicts, and add the file to the repository using the command

git add <name_of_the_file_with_fixed_conflicts>

After all conflicts for a repository are resolved use the command

git commit -m "<new_tag>_conflict_resolved"
to commit the changes (the string after the parameter -m is the comment for the commit and can be any string)

Afterwards the revisions configured in the file .repo/manifests/default.xml should be updated to match the just merged tag.

That's it - not a big deal

When everything is merged, recompile the image to check the results:
# init the environment # . build/envsetup.sh # optional: cleanup the out directory # m clean
# optional: create the kernel first # (this is necessary in my build environment to build the image for OmniROM 14 ... don't ask me why) # lunch omni_zenfone8-user mka kernel # create the image # brunch omni_zenfone8-user

To simplify the process of applying an Android Security Patch to a local OmniROM repository, I wrote a small script:get_aosp_patches.sh
(get_aosp_patches.sh is currently somehow a quick and dirty script and I may rewrite it in the future - but nevertheless it works)

The usage for the script is:

xtrnaw7@fedora01:/data/develop/android$ ./get_aosp_patches.sh -h get_aosp_patches.sh 1.0.0.0 - add an Android security patch to the local repos with a custom ROM Usage: ./get_aosp_patches.sh.sh [new_tag] [yes]xtrnaw7@fedora01:/data/develop/android$

All parameter are optional.

The script supports some environment variables:

Environment variable	Used for	Comment
NEW_TAG	new tag to apply (will be overwritten by the parameter of the script if any)	If no new tag is defined in this variable or in the script parameters, the script will ask the user
YES	do not ask the user for confirmation if not empty	If this variable is not set and the "yes" parameter is not used, the script will ask the user for confirmation before starting the merge.
VERBOSE	print some more messages if this variable is not empty
GIT_PARAMETER	additional parmeter for the command git	Do NOT use "-q" here because the script analyzes the messages written by git!

Before the script get_aosp_patches.sh can be used for the first time, a few values must be adjusted in the script:

# default tag to apply # DEFAULT_NEW_TAG="" # uncomment the next line to use a default new tag # #DEFAULT_NEW_TAG="android-14.0.0_r21" .... # current branch in the local repositories # BRANCH="android-14.0" NEW_TAG="${NEW_TAG:=${DEFAULT_NEW_TAG}}" # default parameter for the git command # GIT_PARAMETER="${GIT_PARAMETER:=}" # path to the local repositories # ROM_PATH="/data/develop/android/OmniROM_14.0"

The script was written for the OmnIROM -- for other Custom ROMs these values must also be corrected:

# name of manifest file with the AOSP repositories # REPO_XML_FILE_WITH_THE_AOSP_REPOSITORIES="omni-aosp.xml" # name of the remote repository in the manifest # REMOTE_NAME="omnirom"

Notes

Download the script from here: get_aosp_patches.sh (Note: The script was updated to fix some errors on 07.02.2024)
An example output of the script is here.

History of this entry

14.07.2024

Added infos about the script to apply an Android Security Patchlevel for the StatixOS repositories

07.02.2024

added more details about how to use the official script for applying a security patch

04.04.2024

added infos about some small changes after QPR2 was released
moved the less important infos about applying an Android Security Patch level to a local repository tree to the separate section Observations while checking how to apply the Android Security patches to a local repository

How to enable adb via WiFi

URL: https://xdaforums.com/t/how-to-enable-adb-via-wifi.4651610/

How to enable adb via WiFi

The Android OS supports adb connections via USB and also via WiFi.

Sometimes it's usefull to enable adb via WiFi in a script. So lets check how that can be done.

To enable adb via WiFi in the Android GUI on the phone just open the Developer Options and activate "Wireless debugging".

After enabling adb via WiFi in the Android GUI the adb connections from machines in the current WLAN must be confirmed in a popup dialog on the phone (if not already done for the current WLAN)

adb via WiFi is then immediately enabled (tick "Always allow on this network" to make the confirmation persistent).

Be aware that the port used for adb via WiFi when enabled in the Android GUI is dynamically selected by the Android OS and is therefor different each time the adb via WiFi is enabled using this method.

The IP address and the port used for adb are visible in the "Wireless debugging" details in the Android GUI

The command to connect using adb via WiFi is then:
[ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ adb -e -L tcp:localhost:5237 connect 192.168.1.148:40719 * daemon not running; starting now at tcp:localhost:5237 * daemon started successfully connected to 192.168.1.148:40719 [ OmniRom 14 Dev - xtrnaw7@t15g ~ ]]

Now a shell via WiFi adb can be opened using this command:

[ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ adb -e -L tcp:localhost:5237 shell ASUS_I006D:/ $ uname -a Linux localhost 5.4.147-Omni-qgki-perf-g736dda6e5ae7 #29 SMP PREEMPT Sun Dec 3 09:59:38 CET 2023 aarch64 Toybox ASUS_I006D:/ $ id uid=2000(shell) gid=2000(shell) groups=2000(shell),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),1078(ext_data_rw),1079(ext_obb_rw),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats),3009(readproc),3011(uhid),3012(readtracefs) context=u:r:shell:s0 ASUS_I006D:/ $Notes:

192.168.1.148 is the IP from the Wireless Debugging details and 40719 is the port from the Wireless Debugging details.

5237 is the local port used; that can be any unused port on your PC greater then 1024.

The computer trying to connect to the phone using adb via WiFi must already be authorized for adb via USB.

The BSSID of the WLANs authorized for adb via WiFi is stored in the adbd keystore, this is the file adb_temp_keys.xml in the directory /data/misc/adb :

# root access is necessary to read the adbd keystore # ASUS_I006D:/ $ su - ASUS_I006D:/ # id uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 ASUS_I006D:/ # ASUS_I006D:/ # cd /data/misc/adb ASUS_I006D:/data/misc/adb # # the adbd keystore is in binary xml format and must be converted to plain xml to read the contents # ASUS_I006D:/data/misc/adb # abx2xml adb_temp_keys.xml adb_temp_keys.xml.plain ASUS_I006D:/data/misc/adb # ASUS_I006D:/data/misc/adb # cat adb_temp_keys.xml.plain <?xml version='1.0' encoding='UTF-8' standalone='yes' ?> <keyStore version="1"> <adbKey key="QAAAAAO//XxV3DN2zq7j8w0uc/kHyC9vDeQt+Vx3IEcxRzdidhIuO0V6YhV9wHQO+Fz5qbC+nHte0X3eQxPGkbAYlNgmK/e9OWu0ccO5ena0vNLO03YfIAcKQ4q4XZZTuUMgtC9g5BVrOInNAfcQ9UwSAP9dWCuvnuCvK3FR6DTDRWl+D9DiP5/4eFsHWFKOBvegn4EhsJLMsM4Yj1r3mrqpmTBvYNDI4W44FPAd9BfGSE4oZ51yDQdoAUVpvNpG/samU8fWwQqbFQTvHFONB3V7+cjHrUi7pKY5fUKTENiBEdmW0qeL0xbfktpbBBQMPjcgcPGPDPIwSro1T6F1SO6P9ywUKeu2yfxyzUk1/fsRYoTapkoGs7oNF2N2otaZdNZX4Z7vMR3o9dspBLN9WaLJ/q8Z96+G8pNzNGnmfP7giZsBl+rhiqqp4FLD5l7qCN71FmW2jZt77ovQE/y5lZo8KL0I2UiFmYZc7RpM25t3shBrR7LF9sriqvJN/wG8aDy/puteZO6RifA9zT4Jcpo3QgfCzqMbrSHaVlE3QmApO9RfIO3tZCn7+KSFoLGL/GvCv67q5+dvTTb9oLJ6njxEfZLikRfJe/6eXQ+4osqis2lGlZZuFtjIeSlc4xUb7s2x5h/hyroeUXHFdFnDyKpMcewKppdooJh6sXKv9XCixFEXz3huhgEAAQA= xtrnaw7@t15g.isbs.de" lastConnection="1705393201326" /> <wifiAP bssid="24:4b:fe:f3:02:58" /></keyStore>ASUS_I006D:/data/misc/adb #

adb via WiFi enabled using the Android GUI is again disabled after the next reboot of the phone. But the BSSID for the already authorized WLANs remains in the adbd keystore (if you ticked "Always allow on this network" in the confirmation dialog for the WLAN).

adb via WiFi using this method can also be enabled with the CLI command settings, e.g:

to retrieve the current status of adb via WiFi use:

settings get global adb_wifi_enabled

to enable adb via WiFi use:

settings put global adb_wifi_enabled 1

to disable adb via WiFi use:

settings put global adb_wifi_enabled 0

Note that enabling adb via WiFi using the setting adb_wifi_enabled will popup a dialog on the phone to authorize the adb via WiFi using the current WLAN if not already authorized.

Examples for enabling or disabling adb via WiFi with the cli command settings

# get the current status of adb via WiFi # ASUS_I006D:/ $ settings get global adb_wifi_enabled 0 ASUS_I006D:/ $ # -> 0 = adb via Wifi is disabled # enable adb via WiFi # ASUS_I006D:/ $ settings put global adb_wifi_enabled 1 ASUS_I006D:/ $ # get the current status of adb via WiFi # ASUS_I006D:/ $ settings get global adb_wifi_enabled 1 ASUS_I006D:/ $ # -> 1 = adb via Wifi is disabled # disable adb via WiFi # ASUS_I006D:/ $ settings put global adb_wifi_enabled 0 ASUS_I006D:/ $ # get the current status of adb via WiFi # ASUS_I006D:/ $ settings get global adb_wifi_enabled 0 ASUS_I006D:/ $

The settings command can be used by the normal user to enable or disable adb via WiFi; no root access is necessary to enable adb via WiFi using this method.

But to retrieve the port used for adb via WiFi root access is required:

ASUS_I006D:/ $ su - -c netstat -tnlp | grep adbd | grep LISTEN | awk -F: '{print $4}' 40719 [ ASUS_I006D:/ $

The entry for the WLAN BSSID can also be written to the adbd keystore in a script, but there is no method to get the adbd to re-read the keystore. Restarting the adbd does not help either, as the daemon overwrites the adbd keystore with the values in memory before it stops.

Therefore, activating adb via WiFi with this method in a script without user intervention only works if adb via WiFi has been activated once in the Android GUI.

Notes:

Note that enabling adb via WiFi using the settings command works only reliable if the development settings are enabled

To enable or disable the development settings via cli command use these commands:

to retrieve the current status of the development settings use:

settings get global development_settings_enabled

to enable the development settings use:

settings put global development_setttings_enabled 1

to disable the development settings use:

settings put global development_settings_enabled 0

Note that the settings command can be used by the normal user; no root access is necessary to enable or disable the development settings using this method.

Examples for enabling or disabling the development settings with the cli command settings

# get the current status of the development settings # ASUS_I006D:/ $ settings get global development_settings_enabled 0 ASUS_I006D:/ $ # -> 0 = development settings are disabled # enable the development settings # ASUS_I006D:/ $ settings put global development_settings_enabled 1 ASUS_I006D:/ $ # ge the current status of the development settings # ASUS_I006D:/ $ settings get global development_settings_enabled 1 ASUS_I006D:/ $ # -> 1 = development settings are enabled # disable the development settings # ASUS_I006D:/ $ settings put global development_settings_enabled 0 ASUS_I006D:/ $ # ge the current status of the development settings # ASUS_I006D:/ $ settings get global development_settings_enabled 0 ASUS_I006D:/ $

In the end, this method to enable adb via WiFi is only useful for temporary interactive usage but not for using it in scripts.

Another method to enable adb via WiFi is to set the property persist.adb.tcp.port:

The property persist.adb.tcp.port can be set to a port that the adb daemon on the phone (adbd) will listen on after the next reboot. Any free port greater than 1024 can be used for this purpose.

To check the used ports on the phone use the netstat command:

ASUS_I006D:/ $ su - -c ' netstat -tnlp | egrep "State|LISTEN" ' Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program Name tcp6 0 0 [::]:1024 [::]:* LISTEN 19167/adbd tcp6 0 0 ::ffff:127.0.0.1:41479 [::]:* LISTEN 8156/com.google.android.youtube ASUS_I006D:/ $

The property can only be set by the root user:

ASUS_I006D:/ $ id uid=2000(shell) gid=2000(shell) groups=2000(shell),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),1078(ext_data_rw),1079(ext_obb_rw),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats),3009(readproc),3011(uhid),3012(readtracefs) context=u:r:shell:s0 ASUS_I006D:/ ASUS_I006D:/ $ setprop persist.adb.tcp.port 6666 Failed to set property 'persist.adb.tcp.port' to '6666'. See dmesg for error reason. ASUS_I006D:/ 1|ASUS_I006D:/ $ su - -c setprop persist.adb.tcp.port 6666 ASUS_I006D:/ $ ASUS_I006D:/ $ getprop persist.adb.tcp.port 6666 ASUS_I006D:/ $

After the next reboot adb connections via WiFi works:

[ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ adb -e -L tcp:localhost:5237 connect 192.168.1.148:6666 * daemon not running; starting now at tcp:localhost:5237 * daemon started successfully connected to 192.168.1.148:6666 [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ adb -e -L tcp:localhost:5237 shell ASUS_I006D:/ $ ls ADF acct asdf bin cache d data_mirror dev init linkerconfig metadata odm oem proc sdcard storage system system_ext vendor_dlkm APD apex batinfo bugreports config data debug_ramdisk etc init.environ.rc lost+found mnt odm_dlkm postinstall product second_stage_resources sys system_dlkm vendor ASUS_I006D:/ $

If adb via WiFi is activated using this method, authorization is only required for the computers that want to connect (analogous to the authorization for adb via USB); the authorization for the used WLAN is not necessary.

And this authorization can be configured via script -- see How to enable access via adb on a new installed_OS.

Therefor this method to enable adb via WiFi can be used in scripts .

Notes

Instead of rebooting the phone it's also sufficient to restart the adbd after setting the property to enable adb via WiFi . But in this case all active adb sessions will be killed.

adb via WiFi, which is activated via the persist.adb.tcp.port property, only works if adb via USB is also activated and already authorized -- either via the Android GUI or by creating the adbd keystore files with a script.

Invalid values for the property persist.adb.tcp.port will be ignored and adb via WiFi will not be activated

Update 14.07.2024

A Magisk Module to automatically enable adb via WiFi can be downloaded from here: https://github.com/mrh929/magisk-wifiadb

General Notes

Both methods to enable adb via WiFi can be used in parallel.

It is possible to use an adb authorization key to connect from several computers in parallel using adb via WiFi; simply copy the file $HOME/.android/adbkey to the other computers.

Test Environments

OmniROM 13
OmniROM 14
ASUS Android 12
ASUS Android 13

How to compile Magisk

URL: not yet published

Note: As of 12.02.2024 this section is still work in progress.

How to compile Magisk

Compiling Magisk is a simple task : Just follow the instructions from this page: https://topjohnwu.github.io/Magisk/build.html

Well, that's the theorie - - now we start with the practice ...

Note: The following instructions are copied from this page and provided with additional information; all instructions are for the Linux OS.

To compile Magisk perform the following tasks:

1. install the necessary pre-requisites for building Magisk (python, git, etc) -- see https://topjohnwu.github.io/Magisk/build.html for details

2. install Android Studio

Download and unpack the tar file with the Android Studio installation files from https://developer.android.com/studio

Use the script ./bin/studio.sh from the tar file to install and configure Android Studio (see the fiel Install-Linux-tar.txt from the tar file; the script starts a GUI installation program)

Note that the rest of these instructions assume that Android Studio is installed in the directory /data/develop/android/android_sdk.

The Android SDK Command Line tools are also necessary : The Download link for these tools is on the bottom of the page https://developer.android.com/studio

To install the Android SDK command line tools just unpack the zip file in the directory with Android Studio.

Afterwards use the executable sdkmanager from the Android SDK Command Line tools to accept all Android SDK licenses:

/data/develop/android/android_sdk$ ./cmdline-tools/bin/sdkmanager --sdk_root=/data/develop/android/android_sdk --licenses

3. create an empty directory for the Magisk source files

4. make the new directory the current working directory and download the Magisk source files using this command:

git clone --recurse-submodules https://github.com/topjohnwu/Magisk.git

5. install the necessary special NDK used for Magisk using the build script from Magisk:

# set the variable used for the directory with the SDK # export ANDROID_SDK_ROOT=/data/develop/android/android_sdk # # download the special NDK for Magisk # ./build.py ndk

(The NDK will be installed in the directory /data/develop/android/android_sdk/ndk/magisk )

6. compile Magisk using the build script from Magisk:
# set the variable used for the directory with the SDK # export ANDROID_SDK_ROOT=/data/develop/android/android_sdk ./build.py all

Example output:

./build.py all

xtrnaw7@fedora01:/data/develop/android/Magisk/Magisk_v27.0$ time ./build.py all * Building binaries: magisk magiskinit magiskboot magiskpolicy busybox * Building the Magisk app Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. Note: /data/develop/android/Magisk/Magisk_v27.0/stub/src/main/java/com/topjohnwu/magisk/DelegateComponentFactory.java uses unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. Output: out/app-debug.apk real 7m42.892s user 23m45.961s sys 2m7.967s xtrnaw7@fedora01:/data/develop/android/Magisk/Magisk_v27.0$

With the default config the script will create a debug version of Magisk. Use the parameter -r to compile Magisk in release mode.

The script build.py can also be used to compile only parts of Magisk:

./build.py --help

xtrnaw7@fedora01:/data/develop/android/Magisk/Magisk$ ./build.py --help usage: build.py [-h] [-r] [-v] [-c CONFIG] {all,binary,cargo,app,stub,emulator,avd_patch,clean,ndk} ... Magisk build script options: -h, --help show this help message and exit -r, --release compile in release mode -v, --verbose verbose output -c CONFIG, --config CONFIG custom config file (default: config.prop) actions: {all,binary,cargo,app,stub,emulator,avd_patch,clean,ndk} all build everything binary build binaries cargo run cargo with proper environment app build the Magisk app stub build the stub app emulator setup AVD for development avd_patch patch AVD ramdisk.img clean cleanup ndk setup Magisk NDK xtrnaw7@fedora01:/data/develop/android/Magisk/Magisk$

In the end, compiling Magisk is a simple task if you fulfill the requirements.

Notes:

As of 02/04/2024, the source files for Magisk v27.0 occupy around 350 MB.
After compiling the image, the files for Magisk v27.0 take up about 2.3 GB.
The Android SDK and NDK occupy around 2.7 GB.

The tar/zip files with the sources available from the Magisk Github repository page https://github.com/topjohnwu/Magisk/releases do not contain the files required to compile the external modules used to compile Magisk.
Therefore, the compilation process will fail if only the source files from these tar/zip files are used

All parts of Magisk installed on a phone must come from one source of Magisk - for example, you cannot mix self-compiled Magisk files with files from the original Magisk app.

Self-compiled Magisk apks are only useful for testing and developing -- For usage on productive phones the official Magisk apks should be used

Some hints for repo sync troubleshooting

URL: https://xdaforums.com/t/some-hints-for-trouble-shooting-repo-sync-issues.4655310/

Some hints for repo sync troubleshooting

The tool repo is used to synchronize local repositories for a custom Android ROM with the repositories used by the maintainer of the custom ROM.
(See https://source.android.com/docs/setup/reference/repo for usage details for the tool repo.)

In this post I collect the error messages I received while using the tool repo to synchronize the source for the custom ROM OmniROM and how I solved the problems..

I use a Linux installation (Fedora) to build the custom Android ROM images, but I suspect these errors may also occur in other build environments.

error: command 'sync' requires repo to be installed first

Error:

xtrnaw7@fedora01:/data/develop/android$ repo sync error: command 'sync' requires repo to be installed first. Use "repo init" to install it here. xtrnaw7@fedora01:/data/develop/android

Solution:

"repo sync" must be executed in the directory with the source files for the Custom ROM

repo sync fails with error code 429

Error:

The command repo sync fails with an error message like this:

platform/external/rust/crates/instant: error: RPC failed; HTTP 429 curl 22 The requested URL returned error: 429 fatal: expected 'acknowledgments'

Solution:

Restart the command repo sync again some minutes later. The error 429 is in most cases only a temporary error.

repo sync fails for unkown reason

Error

The command repo sync fails for unknown reason

Solution:

In this case, it is always a good idea to restart the "repo sync" with the parameter --force-sync:

repo sync --force-sync

If that doesn't help, I recommend waiting a day and repeating the synchronization of the project repositories - there may be ongoing updates to the project repositories that result in temporarily inconsistent project repositories.

A new version of repo is available

Error:

xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ time repo sync --force-sync Fetching: 100% (1264/1264), done in 2m15.576s info: A new version of repo is availablewarning: repo is not tracking a remote branch, so it will not receive updates ================================================================================
Solution:

This is not an error but it's recommended to update the repo script:

curl https://storage.googleapis.com/git-repo-downloads/repo >/tmp/repo && cat /tmp/repo >$( which repo )

GitCommandError: 'reset --keep v2.41^0' on repo failed

Error:xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ repo sync Fetching: 100% (1264/1264), done in 8m2.961s info: A new version of repo is available warning: repo is not tracking a remote branch, so it will not receive updates ================================================================================ Repo command failed: RepoUnhandledExceptionError GitCommandError: 'reset --keep v2.41^0' on repo failed stderr: error: Entry 'docs/python-support.md' not uptodate. Cannot merge. fatal: Could not reset index file to revision 'v2.41^0'. xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$

Solution:

Synchronise the directory .repo/repo manually:

xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ cd .repo/repo/ xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0/.repo/repo$ git pull Updating 4c80921..4217a82 Fast-forward docs/python-support.md | 5 ++--- git_trace2_event_log_base.py | 3 +++ manifest_xml.py | 4 ++-- project.py | 84 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++--------------------------- repo | 41 +++++------------------------------------ subcmds/sync.py | 4 ++-- tests/conftest.py | 9 +++++++++ tests/test_manifest_xml.py | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 8 files changed, 127 insertions(+), 70 deletions(-) xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0/.repo/repo$

History of this entry

04.02.2024

initial release

05.02.2024

updated the section about failures for unkown reasons

How to get the application name and version from an apk file

URL: https://xdaforums.com/t/how-to-get-the-application-name-and-version-from-an-apk-file.4654732/

How to get the application name and version from an apk file

The name and version of the application in an apk file is part of the metadata . The tool aapt (Android Asset Packaging Tool) can be used to read the meta data of an apk file.

e.g:

aapt dump badging org.omnirom.chromium_v535910915.apk

[ xtrnaw7@t15g /data/backup/Android/EssentialApps ] $ aapt dump badging org.omnirom.chromium_v535910915.apk package: name='org.omnirom.chromium' versionCode='535910915' versionName='108.0.5359.109' platformBuildVersionName='13' platformBuildVersionCode='33' compileSdkVersion='33' compileSdkVersionCodename='13' sdkVersion:'23' targetSdkVersion:'33' uses-permission: name='android.permission.ACCESS_COARSE_LOCATION' uses-permission: name='android.permission.ACCESS_FINE_LOCATION' uses-permission-sdk-23: name='android.permission.ACCESS_WIFI_STATE' uses-permission: name='android.permission.ACCESS_NETWORK_STATE' uses-permission-sdk-23: name='android.permission.BLUETOOTH_ADMIN' maxSdkVersion='30' uses-permission-sdk-23: name='android.permission.BLUETOOTH_CONNECT' uses-permission-sdk-23: name='android.permission.BLUETOOTH_ADVERTISE' uses-permission-sdk-23: name='android.permission.BLUETOOTH' uses-permission-sdk-23: name='android.permission.BLUETOOTH_SCAN' uses-permission-sdk-23: name='android.permission.REORDER_TASKS' uses-permission-sdk-23: name='android.permission.REQUEST_INSTALL_PACKAGES' uses-permission: name='android.permission.CAMERA' uses-permission: name='android.permission.DOWNLOAD_WITHOUT_NOTIFICATION' uses-permission: name='android.permission.FOREGROUND_SERVICE' uses-permission: name='android.permission.INTERNET' uses-permission: name='android.permission.MODIFY_AUDIO_SETTINGS' uses-permission: name='android.permission.NFC' uses-permission: name='android.permission.POST_NOTIFICATIONS' uses-permission: name='android.permission.QUERY_ALL_PACKAGES' uses-permission: name='android.permission.READ_EXTERNAL_STORAGE' uses-permission-sdk-23: name='android.permission.READ_MEDIA_AUDIO' uses-permission-sdk-23: name='android.permission.READ_MEDIA_IMAGES' uses-permission-sdk-23: name='android.permission.READ_MEDIA_VIDEO' uses-permission: name='android.permission.RECEIVE_BOOT_COMPLETED' uses-permission: name='android.permission.RECORD_AUDIO' uses-permission-sdk-23: name='android.permission.USE_BIOMETRIC' uses-permission-sdk-23: name='android.permission.USE_FINGERPRINT' uses-permission: name='android.permission.VIBRATE' uses-permission: name='android.permission.WAKE_LOCK' uses-permission: name='android.permission.WRITE_EXTERNAL_STORAGE' uses-permission: name='org.omnirom.chromium.permission.READ_WRITE_BOOKMARK_FOLDERS' uses-permission: name='org.omnirom.chromium.TOS_ACKED' uses-permission: name='com.chrome.permission.DEVICE_EXTRAS' uses-permission: name='com.android.launcher.permission.INSTALL_SHORTCUT' application-label:'Chromium' application-label-af:'Chromium' application-label-am:'Chromium' application-label-ar:'Chromium' application-label-as:'Chromium' application-label-az:'Chromium' application-label-be:'Chromium' application-label-bg:'Chromium' application-label-bn:'Chromium' application-label-bs:'Chromium' application-label-ca:'Chromium' application-label-cs:'Chromium' application-label-da:'Chromium' application-label-de:'Chromium' application-label-el:'Chromium' application-label-en-GB:'Chromium' application-label-en-US:'Chromium' application-label-es:'Chromium' application-label-es-US:'Chromium' application-label-et:'Chromium' application-label-eu:'Chromium' application-label-fa:'Chromium' application-label-fi:'Chromium' application-label-fr:'Chromium' application-label-fr-CA:'Chromium' application-label-gl:'Chromium' application-label-gu:'Chromium' application-label-hi:'Chromium' application-label-hr:'Chromium' application-label-hu:'Chromium' application-label-hy:'Chromium' application-label-in:'Chromium' application-label-is:'Chromium' application-label-it:'Chromium' application-label-iw:'Chromium' application-label-ja:'Chromium' application-label-ka:'Chromium' application-label-kk:'Chromium' application-label-km:'Chromium' application-label-kn:'Chromium' application-label-ko:'Chromium' application-label-ky:'Chromium' application-label-lo:'Chromium' application-label-lt:'Chromium' application-label-lv:'Chromium' application-label-mk:'Chromium' application-label-ml:'Chromium' application-label-mn:'Chromium' application-label-mr:'Chromium' application-label-ms:'Chromium' application-label-my:'Chromium' application-label-nb:'Chromium' application-label-ne:'Chromium' application-label-nl:'Chromium' application-label-or:'Chromium' application-label-pa:'Chromium' application-label-pl:'Chromium' application-label-pt:'Chromium' application-label-pt-BR:'Chromium' application-label-pt-PT:'Chromium' application-label-ro:'Chromium' application-label-ru:'Chromium' application-label-si:'Chromium' application-label-sk:'Chromium' application-label-sl:'Chromium' application-label-sq:'Chromium' application-label-sr:'Chromium' application-label-sr-Latn:'Chromium' application-label-sv:'Chromium' application-label-sw:'Chromium' application-label-ta:'Chromium' application-label-te:'Chromium' application-label-th:'Chromium' application-label-tl:'Chromium' application-label-tr:'Chromium' application-label-uk:'Chromium' application-label-ur:'Chromium' application-label-uz:'Chromium' application-label-vi:'Chromium' application-label-zh-CN:'Chromium' application-label-zh-HK:'Chromium' application-label-zh-TW:'Chromium' application-label-zu:'Chromium' application-icon-160:'res/drawable-v26/ic_launcher.xml' application-icon-240:'res/drawable-v26/ic_launcher.xml' application-icon-320:'res/drawable-v26/ic_launcher.xml' application-icon-480:'res/drawable-v26/ic_launcher.xml' application-icon-640:'res/drawable-v26/ic_launcher.xml' application-icon-65534:'res/drawable-v26/ic_launcher.xml' application-icon-65535:'res/drawable-v26/ic_launcher.xml' application: label='Chromium' icon='res/drawable-v26/ic_launcher.xml' feature-group: label='' uses-gl-es: '0x20000' uses-feature-not-required: name='android.hardware.camera' uses-feature-not-required: name='android.hardware.camera.autofocus' uses-feature-not-required: name='android.hardware.location.gps' uses-feature-not-required: name='android.hardware.microphone' uses-feature-not-required: name='android.hardware.touchscreen' uses-feature-sdk-23: name='android.hardware.bluetooth' uses-implied-feature-sdk-23: name='android.hardware.bluetooth' reason='requested android.permission.BLUETOOTH permission, requested android.permission.BLUETOOTH_ADMIN permission, and targetSdkVersion > 4' uses-feature: name='android.hardware.location' uses-implied-feature: name='android.hardware.location' reason='requested android.permission.ACCESS_COARSE_LOCATION permission, and requested android.permission.ACCESS_FINE_LOCATION permission' uses-feature: name='android.hardware.screen.portrait' uses-implied-feature: name='android.hardware.screen.portrait' reason='one or more activities have specified a portrait orientation' uses-feature-sdk-23: name='android.hardware.wifi' uses-implied-feature-sdk-23: name='android.hardware.wifi' reason='requested android.permission.ACCESS_WIFI_STATE permission' provides-component:'app-widget' provides-component:'search' other-activities other-receivers other-services supports-screens: 'small' 'normal' 'large' 'xlarge' supports-any-density: 'true' locales: '--_--' 'af' 'am' 'ar' 'as' 'az' 'be' 'bg' 'bn' 'bs' 'ca' 'cs' 'da' 'de' 'el' 'en-GB' 'en-US' 'es' 'es-US' 'et' 'eu' 'fa' 'fi' 'fr' 'fr-CA' 'gl' 'gu' 'hi' 'hr' 'hu' 'hy' 'in' 'is' 'it' 'iw' 'ja' 'ka' 'kk' 'km' 'kn' 'ko' 'ky' 'lo' 'lt' 'lv' 'mk' 'ml' 'mn' 'mr' 'ms' 'my' 'nb' 'ne' 'nl' 'or' 'pa' 'pl' 'pt' 'pt-BR' 'pt-PT' 'ro' 'ru' 'si' 'sk' 'sl' 'sq' 'sr' 'sr-Latn' 'sv' 'sw' 'ta' 'te' 'th' 'tl' 'tr' 'uk' 'ur' 'uz' 'vi' 'zh-CN' 'zh-HK' 'zh-TW' 'zu' densities: '160' '240' '320' '480' '640' '65534' '65535' native-code: 'arm64-v8a' [ xtrnaw7@t15g /data/backup/Android/EssentialApps ] $

To get the application name from an apk file use

aapt dump badging <apk_file> | sed -n "s/^application-label:'$.*$'/\1/p"

To get the application version from an apk file use

aapt dump badging <apk_file> | grep "versionName=" | cut -d"'" -f6

Examples

[ xtrnaw7@t15g /data/backup/Android/EssentialApps ] $ aapt dump badging org.omnirom.chromium_v535910915.apk | sed -n "s/^application-label:'$.*$'/\1/p" Chromium [ xtrnaw7@t15g /data/backup/Android/EssentialApps ] $ [ xtrnaw7@t15g /data/backup/Android/EssentialApps ] $ [ xtrnaw7@t15g /data/backup/Android/EssentialApps ] $ aapt dump badging org.omnirom.chromium_v535910915.apk | grep "versionName=" | cut -d"'" -f6 108.0.5359.109 [ xtrnaw7@t15g /data/backup/Android/EssentialApps ] $

aapt is part of the Android SDK; the Android SDK can be downloaded from here https://developer.android.com/studio

A shell script for bash to rename apk files might look like:

rename_apk.sh

#!/bin/bash # # rename_apk.sh - rename apk files using the application name and version from the within the apk # # Usage: Usage: %0 [apkfile1] [...[apkfile#]]" # # The script uses the executable "aapt" to read the meta data of the apk files. # The executable "aapt" must be available via the PATH or via the variable AAPT # # History # 05.02.2024 v1.0.0.0 /bs # initial release # # uncomment the line enable dry-run mode (or set the variable PREFIX to "echo" before starting the script) # # PREFIX="echo" if [ "${PREFIX}"x != ""x ] ; then echo "" echo "-----------------------------------------------------------------------" echo "INFO: The script is running dry-run mode -- no filename will be changed" echo "-----------------------------------------------------------------------" echo "" fi # search the aapt executable # AAPT="${AAPT:=$( which aapt )}" if [ "$1"x = ""x -o "$1"x = "-h"x -o "$1"x = "--help"x ] ; then echo "Usage: %0 [apkfile1] [...[apkfile#]]" echo "(set the environment variable \"PREFIX\" to \"echo\" before starting the script for dry-run mode)" exit 1 fi if [ "${AAPT}"x = ""x ] ; then echo "ERROR: aapt executable not found in the path and the variable AAPT is empty" exit 5 fi while [ $# -ne 0 ] ; do echo "" CUR_APK_FILE="$1" shift echo "Processing the apk file \"${CUR_APK_FILE}\" ..." CUR_APP_NAME="$( "${AAPT}" dump badging "${CUR_APK_FILE}" | sed -n "s/^application-label:'$.*$'/\1/p" )" CUR_APP_VERSION="$( "${AAPT}" dump badging ""${CUR_APK_FILE}"" | grep "versionName=" | cut -d"'" -f6 )" echo " The application name is \"${CUR_APP_NAME}\" " echo " The application version is \"${CUR_APP_VERSION}\" " if [ "${CUR_APP_NAME}"x = ""x ] ; then echo "ERROR: Can not read the application name for the apk fle \"${CUR_APK_FILE}\" " continue fi if [ "${CUR_APP_VERSION}"x = ""x ] ; then echo "WARNING: Can not read the application version for the apk fle \"${CUR_APK_FILE}\" - now using 1.0" CUR_APP_VERSION="1.0" fi [[ ${CUR_APP_VERSION} != v* ]] && CUR_APP_VERSION="v${CUR_APP_VERSION}" NEW_APK_FILE_NAME="$( echo "${CUR_APP_NAME}" | tr " " "_" )_${CUR_APP_VERSION}.apk" echo "Now renaming \"${CUR_APK_FILE}\" to \"${NEW_APK_FILE_NAME}\" ..." ${PREFIX} mv "${CUR_APK_FILE}" "${NEW_APK_FILE_NAME}" done if [ "${PREFIX}"x != ""x ] ; then echo "" echo "-----------------------------------------------------------------------" echo "INFO: The script was running in dry-run mode -- no filename was changed" echo "-----------------------------------.-----------------------------------" echo "" fi

The usage of the script is

[ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/EssentialApps ] $ /data/develop/android/scripts/rename_apk.sh -h Usage: %0 [apkfile1] [...[apkfile#]] (set the environment variable "PREFIX" to "echo" before starting the script for dry-run mode) [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/EssentialApps ] $

Sample output of the script in dry-run mode

[ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/EssentialApps ] $ PREFIX="echo" /data/develop/android/scripts/rename_apk.sh *apk ----------------------------------------------------------------------- INFO: The script is running dry-run mode -- no filename will be changed ----------------------------------------------------------------------- Processing the apk file "axp.tool.apkextractor_v5.apk" ... The application name is "Apk Extractor" The application version is "1.4" Now renaming "axp.tool.apkextractor_v5.apk" to "Apk_Extractor_v1.4.apk" ... mv axp.tool.apkextractor_v5.apk Apk_Extractor_v1.4.apk Processing the apk file "com.android.keepass_196.apk" ... The application name is "KeePassDroid" The application version is "2.5.13" Now renaming "com.android.keepass_196.apk" to "KeePassDroid_v2.5.13.apk" ... mv com.android.keepass_196.apk KeePassDroid_v2.5.13.apk Processing the apk file "com.apk.editor_v25.apk" ... The application name is "APK Explorer & Editor" The application version is "v0.25" Now renaming "com.apk.editor_v25.apk" to "APK_Explorer_&_Editor_v0.25.apk" ... mv com.apk.editor_v25.apk APK_Explorer_&_Editor_v0.25.apk Processing the apk file "com.bytehamster.changelog_v49.apk" ... The application name is "Changelog" The application version is "4.1" Now renaming "com.bytehamster.changelog_v49.apk" to "Changelog_v4.1.apk" ... mv com.bytehamster.changelog_v49.apk Changelog_v4.1.apk Processing the apk file "com.dkanada.openapk_v35.apk" ... The application name is "OpenAPK" The application version is "1.2.1" Now renaming "com.dkanada.openapk_v35.apk" to "OpenAPK_v1.2.1.apk" ... mv com.dkanada.openapk_v35.apk OpenAPK_v1.2.1.apk Processing the apk file "com.elsdoerfer.android.autostarts_v33.apk" ... The application name is "Autostarts" The application version is "2.0.0" Now renaming "com.elsdoerfer.android.autostarts_v33.apk" to "Autostarts_v2.0.0.apk" ... mv com.elsdoerfer.android.autostarts_v33.apk Autostarts_v2.0.0.apk Processing the apk file "com.keramidas.TitaniumBackupAddon_v1.apk" ... The application name is "Titanium Backup Add-on" The application version is "1.0.0" Now renaming "com.keramidas.TitaniumBackupAddon_v1.apk" to "Titanium_Backup_Add-on_v1.0.0.apk" ... mv com.keramidas.TitaniumBackupAddon_v1.apk Titanium_Backup_Add-on_v1.0.0.apk Processing the apk file "com.keramidas.TitaniumBackup_v417.apk" ... The application name is "Titanium Backup" The application version is "8.4.0.2" Now renaming "com.keramidas.TitaniumBackup_v417.apk" to "Titanium_Backup_v8.4.0.2.apk" ... mv com.keramidas.TitaniumBackup_v417.apk Titanium_Backup_v8.4.0.2.apk Processing the apk file "com.kunzisoft.keepass.libre_57.apk" ... The application name is "KeePassDX" The application version is "2.9.13" Now renaming "com.kunzisoft.keepass.libre_57.apk" to "KeePassDX_v2.9.13.apk" ... mv com.kunzisoft.keepass.libre_57.apk KeePassDX_v2.9.13.apk Processing the apk file "com.matoski.adbm_v27.apk" ... The application name is "ADB Manager" The application version is "1.2.2" Now renaming "com.matoski.adbm_v27.apk" to "ADB_Manager_v1.2.2.apk" ... mv com.matoski.adbm_v27.apk ADB_Manager_v1.2.2.apk Processing the apk file "com.offsec.nhterm_2020040001.apk" ... The application name is "NetHunter Terminal" The application version is "2020.4-RC1" Now renaming "com.offsec.nhterm_2020040001.apk" to "NetHunter_Terminal_v2020.4-RC1.apk" ... mv com.offsec.nhterm_2020040001.apk NetHunter_Terminal_v2020.4-RC1.apk Processing the apk file "com.termux_v118.apk" ... The application name is "Termux" The application version is "0.118.0" Now renaming "com.termux_v118.apk" to "Termux_v0.118.0.apk" ... mv com.termux_v118.apk Termux_v0.118.0.apk Processing the apk file "F-Droid.apk" ... The application name is "F-Droid" The application version is "1.10" Now renaming "F-Droid.apk" to "F-Droid_v1.10.apk" ... mv F-Droid.apk F-Droid_v1.10.apk Processing the apk file "FoxMmm-0.5.6.apk" ... The application name is "Fox's Magisk Module Manager" The application version is "0.5.6" Now renaming "FoxMmm-0.5.6.apk" to "Fox's_Magisk_Module_Manager_v0.5.6.apk" ... mv FoxMmm-0.5.6.apk Fox's_Magisk_Module_Manager_v0.5.6.apk Processing the apk file "io.github.muntashirakon.AppManager_v397.apk" ... The application name is "App Manager" The application version is "2.6.5.1" Now renaming "io.github.muntashirakon.AppManager_v397.apk" to "App_Manager_v2.6.5.1.apk" ... mv io.github.muntashirakon.AppManager_v397.apk App_Manager_v2.6.5.1.apk Processing the apk file "MiX.addon.Archive-arm64.apk" ... The application name is "MiX Archive" The application version is "3.13" Now renaming "MiX.addon.Archive-arm64.apk" to "MiX_Archive_v3.13.apk" ... mv MiX.addon.Archive-arm64.apk MiX_Archive_v3.13.apk Processing the apk file "MiX.addon.AutoTag.apk" ... The application name is "MiX AutoTag" The application version is "1.0" Now renaming "MiX.addon.AutoTag.apk" to "MiX_AutoTag_v1.0.apk" ... mv MiX.addon.AutoTag.apk MiX_AutoTag_v1.0.apk Processing the apk file "MiX.addon.Codecs-arm64.apk" ... The application name is "MiX Codecs" The application version is "2.5" Now renaming "MiX.addon.Codecs-arm64.apk" to "MiX_Codecs_v2.5.apk" ... mv MiX.addon.Codecs-arm64.apk MiX_Codecs_v2.5.apk Processing the apk file "MiX.addon.Image-arm64.apk" ... The application name is "MiX Image" The application version is "2.10" Now renaming "MiX.addon.Image-arm64.apk" to "MiX_Image_v2.10.apk" ... mv MiX.addon.Image-arm64.apk MiX_Image_v2.10.apk Processing the apk file "MiX.addon.Metadata.apk" ... The application name is "MiX Metadata" The application version is "1.11" Now renaming "MiX.addon.Metadata.apk" to "MiX_Metadata_v1.11.apk" ... mv MiX.addon.Metadata.apk MiX_Metadata_v1.11.apk Processing the apk file "MiX.addon.PDF-arm64.apk" ... The application name is "MiX PDF" The application version is "1.13" Now renaming "MiX.addon.PDF-arm64.apk" to "MiX_PDF_v1.13.apk" ... mv MiX.addon.PDF-arm64.apk MiX_PDF_v1.13.apk Processing the apk file "MiX.addon.Signer.apk" ... The application name is "MiX Signer" The application version is "1.2" Now renaming "MiX.addon.Signer.apk" to "MiX_Signer_v1.2.apk" ... mv MiX.addon.Signer.apk MiX_Signer_v1.2.apk Processing the apk file "MiX.addon.SMB.apk" ... The application name is "MiX SMB" The application version is "2.1" Now renaming "MiX.addon.SMB.apk" to "MiX_SMB_v2.1.apk" ... mv MiX.addon.SMB.apk MiX_SMB_v2.1.apk Processing the apk file "MiX.addon.Tagger.apk" ... The application name is "MiX Tagger" The application version is "1.4" Now renaming "MiX.addon.Tagger.apk" to "MiX_Tagger_v1.4.apk" ... mv MiX.addon.Tagger.apk MiX_Tagger_v1.4.apk Processing the apk file "MiXplorer.v6.63.1.apk" ... The application name is "MiXplorer" The application version is "6.63.1" Now renaming "MiXplorer.v6.63.1.apk" to "MiXplorer_v6.63.1.apk" ... mv MiXplorer.v6.63.1.apk MiXplorer_v6.63.1.apk Processing the apk file "NetHunterStore.apk" ... The application name is "NetHunter Store" The application version is "2019.3" Now renaming "NetHunterStore.apk" to "NetHunter_Store_v2019.3.apk" ... mv NetHunterStore.apk NetHunter_Store_v2019.3.apk Processing the apk file "Notecase_Pro_1.2.2.apk" ... The application name is "NoteCase Pro" The application version is "1.2.2" Now renaming "Notecase_Pro_1.2.2.apk" to "NoteCase_Pro_v1.2.2.apk" ... mv Notecase_Pro_1.2.2.apk NoteCase_Pro_v1.2.2.apk Processing the apk file "OmniStore.apk" ... The application name is "OmniStore" The application version is "2.5.17" Now renaming "OmniStore.apk" to "OmniStore_v2.5.17.apk" ... mv OmniStore.apk OmniStore_v2.5.17.apk Processing the apk file "org.omnirom.chromium_v535910915.apk" ... The application name is "Chromium" The application version is "108.0.5359.109" Now renaming "org.omnirom.chromium_v535910915.apk" to "Chromium_v108.0.5359.109.apk" ... mv org.omnirom.chromium_v535910915.apk Chromium_v108.0.5359.109.apk Processing the apk file "yasnac-v1.1.5.r65.15110ef310-release.apk" ... The application name is "Yet Another SafetyNet Attestation Checker" The application version is "v1.1.5.r65.15110ef310" Now renaming "yasnac-v1.1.5.r65.15110ef310-release.apk" to "Yet_Another_SafetyNet_Attestation_Checker_v1.1.5.r65.15110ef310.apk" ... mv yasnac-v1.1.5.r65.15110ef310-release.apk Yet_Another_SafetyNet_Attestation_Checker_v1.1.5.r65.15110ef310.apk -------------------------------------------------------------------- INFO: The script was running dry-run mode -- no filename was changed -------------------------------------------------------------------- [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/EssentialApps ] $

The shell script can run on a Linux PC and also in an adb shell on the phone if there is aapt executable available on the phone.

The script assumes that the name of the application is used in a format suitable for file names.

I recommend using the script in dry-run mode before renaming multiple apk files.

The current version of the script rename_apk.sh can be downloaded from here:

rename_apk.sh

Credits

The code to retrieve the application name and version of the application from the apk file is copied from this tool

https://github.com/JiiPlus/APK-File-Renaming-Tool/tree/batch-file-rename

Some hints for fixing errors building an Custom ROM Android image

URL: https://xdaforums.com/t/some-hints-for-fixing-errors-building-an-custom-rom-android-image.4655328/

Some hints for fixing errors building an Custom ROM Android image

In this post I summarize all the errors I have encountered so far when creating a custom ROM Android image (in most cases the OmniROM). I'm using a Linux system with Fedora to build the Android image, but I suspect that the errors listed may also occur in other environments.

I will update this post as soon as I receive a new bug.

A change log of the changes can be found at the end of this entry.

Building the image fails with an error like this: unrecognized property "versions_with_info"

ERROR: unrecognized property "versions_with_info"

If the compile fails with error messages like these:

error: vendor/qcom/opensource/interfaces/wifi/supplicant/aidl/Android.bp:16:23: unrecognized property "versions_with_info" error: vendor/qcom/opensource/interfaces/bluetooth_audio/aidl/Android.bp:17:23: unrecognized property "versions_with_info"
Do a forced repo sync:

repo sync --force-sync

and start the compile process again .

Building the image fails with an error like this: hardware/qcom-caf/sm8350/display/libhistogram/Android.bp:15:1: "libhistogram" depends on undefined module "qti_kernel_headers"

ERROR: .. depends on undefined module ....

If the compile fails with error messages like these:

[100% 1027/1027] analyzing Android.bp files and generating ninja file at out/soong/build.ninja FAILED: out/soong/build.ninja cd "$(dirname "out/host/linux-x86/bin/soong_build")" && BUILDER="$PWD/$(basename "out/host/linux-x86/bin/soong_build")" && cd / && env -i "$BUILDER" --top "$TOP" --soong_out "out/soong" --out "out" -o out/soong/build.ninja --baze l-mode --globListDir build --globFile out/soong/globs-build.ninja -t -l out/.module_paths/Android.bp.list --available_env out/soong/soong.environment.available --used_env out/soong/soong.environment.used.build Android.bp error: hardware/qcom-caf/sm8350/display/libhistogram/Android.bp:15:1: "libhistogram" depends on undefined module "qti_kernel_headers". Module "libhistogram" is defined in namespace "." which can read these 5 namespaces: ["." "." "device/generic/goldfish-opengl" "device/generic/goldfish" "hardware/google/camera/devices/EmulatedCamera"] Module "qti_kernel_headers" can be found in these namespaces: ["hardware/google/pixel" "kernel/asus/sm8350"] Or did you mean ["device_kernel_headers" "kernel_log_monitor" "libkernel_cmdline" "qti_powerhal_headers"]? error: hardware/qcom-caf/sm8350/display/libmemutils/Android.bp:1:1: "libmemutils" depends on undefined module "qti_kernel_headers". Module "libmemutils" is defined in namespace "." which can read these 5 namespaces: ["." "." "device/generic/goldfish-opengl" "device/generic/goldfish" "hardware/google/camera/devices/EmulatedCamera"] Module "qti_kernel_headers" can be found in these namespaces: ["hardware/google/pixel" "kernel/asus/sm8350"] Or did you mean ["device_kernel_headers" "kernel_log_monitor" "libkernel_cmdline" "qti_powerhal_headers"]? 06:33:04 soong bootstrap failed with: exit status 1

Check the first lines about the config in the output of the build tools, e.g.:

... making libs for aosp_arm-userdebug ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=14 PRODUCT_INCLUDE_TAGS=com.android.mainline TARGET_PRODUCT=aosp_arm TARGET_BUILD_VARIANT=userdebug TARGET_ARCH=arm TARGET_ARCH_VARIANT=armv7-a-neonTARGET_CPU_VARIANT=generic HOST_OS=linux HOST_OS_EXTRA=Linux-6.6.14-200.fc39.x86_64-x86_64-Fedora-Linux-39-(MATE-Compiz) HOST_CROSS_OS=windows BUILD_ID=UQ1A.240205.004.B1 OUT_DIR=out ============================================ ...

If the values for TARGET_PRODUCT, TARGET_ARCH, or TARGET_ARCH_VARIANT are not okay, you did not initialize the environment. To fix this error execute these commands:
. ./build/envsetup.sh lunch omni_zenfone8-user

(replace omni_zenfone8-user with your the tag for your phone or use the command brunch to select from a list)

Afterwards check the results using the command printconfig, e.g.:

xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ printconfig ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=14 PRODUCT_INCLUDE_TAGS=com.android.mainline TARGET_PRODUCT=omni_zenfone8 TARGET_BUILD_VARIANT=user TARGET_ARCH=arm64 TARGET_ARCH_VARIANT=armv8-aTARGET_CPU_VARIANT=generic TARGET_2ND_ARCH=arm TARGET_2ND_ARCH_VARIANT=armv8-a TARGET_2ND_CPU_VARIANT=generic HOST_OS=linux HOST_OS_EXTRA=Linux-6.6.14-200.fc39.x86_64-x86_64-Fedora-Linux-39-(MATE-Compiz) HOST_CROSS_OS=windows BUILD_ID=UQ1A.240205.004.B1 OUT_DIR=out ROM_BUILDTYPE=WEEKLY ============================================ xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$

and repeat the failed command.

The function printconfig is defined in the file ./build/envsetup.sh so in case you get this error:

xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ printconfig bash: printconfig: command not found xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$

you did not source in ./build/envsetup.sh.

Building the image fails with an error like this : java.util.zip.ZipException: zip END header not found

ERROR: java.util.zip.ZipException: zip END header not found

If the build of the OmniROM installation image fails with error messages like this:

[ 28% 392/1395] //vendor/asus/zenfone8:AsusGalleryBurst signapk [common] FAILED: out/soong/.intermediates/vendor/asus/zenfone8/AsusGalleryBurst/android_common/signed/AsusGalleryBurst.apk rm -f out/soong/.intermediates/vendor/asus/zenfone8/AsusGalleryBurst/android_common/signed/AsusGalleryBurst.apk && prebuilts/jdk/jdk17/linux-x86/bin/java -XX:OnError="cat hs_err_pid%p.log" -XX:CICompilerCount=6 -XX:+UseDynamicNumberOfGCThreads -Djava.library.path=$(dirname out/host/linux-x86/lib64/libconscrypt_openjdk_jni.so) -jar out/host/linux-x86/framework/signapk.jar build/make/target/product/security/platform.x509.pem build/make/target/product/security/platform.pk8 out/soong/.intermediates/vendor/asus/zenfone8/AsusGalleryBurst/android_common/dex-uncompressed/AsusGalleryBurst.apkout/soong/.intermediates/vendor/asus/zenfone8/AsusGalleryBurst/android_common/signed/AsusGalleryBurst.apk java.util.zip.ZipException: zip END header not found at java.base/java.util.zip.ZipFile$Source.findEND(ZipFile.java:1469) ....

Check the type of the source apk file from the error message:

xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ ls -l ./out/soong/.intermediates/vendor/asus/zenfone8/AsusGalleryBurst/android_common/dex-uncompressed/AsusGalleryBurst.apk -rw-rw-r-- 1 xtrnaw7 xtrnaw7 132 Jan 22 09:40 ./out/soong/.intermediates/vendor/asus/zenfone8/AsusGalleryBurst/android_common/dex-uncompressed/AsusGalleryBurst.apk xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ file ./out/soong/.intermediates/vendor/asus/zenfone8/AsusGalleryBurst/android_common/dex-uncompressed/AsusGalleryBurst.apk ./out/soong/.intermediates/vendor/asus/zenfone8/AsusGalleryBurst/android_common/dex-uncompressed/AsusGalleryBurst.apk: ASCII text xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ cat ./out/soong/.intermediates/vendor/asus/zenfone8/AsusGalleryBurst/android_common/dex-uncompressed/AsusGalleryBurst.apk version https://git-lfs.github.com/spec/v1 oid sha256:99eabd18346f546f38788ea9f64843f7a05fca87b3c1166b65be67f3a81bc722 size 6486605 xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$

These file contents are invalid -- that must be an installable Android apk file.

This error could be caused by a missing git component. To double check, check the file type of the original apk file in your local repositories:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ file vendor/asus/zenfone8/proprietary/priv-app/AsusGallery/AsusGallery.apk vendor/asus/zenfone8/proprietary/priv-app/AsusGallery/AsusGallery.apk: ASCII text [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

In this case most probably the git command line extension git lfs is missing, use this command to check that:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ git lfs git: 'lfs' is not a git command. See 'git --help'. The most similar command is log [ OmniRom 14 Dev - xtrnaw7@t15g

To fix the issue install git lfs - in Fedora this can be done using this command:
sudo dnf install git-lfs

Afterwards delete all files in the directory vendor/asus in your local repository and re-sync again to fetch the correct files:

cd <local_omnirom14_repository> rm -rf vendor/asus/../../.repo/projects/vendor/asus.git rm -rf vendor/asus/.git vendor/asus/.gitattributes vendor/asus # # fetch the ASUS packages again # repo sync

Check the result

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ file vendor/asus/zenfone8/proprietary/priv-app/AsusGallery/AsusGallery.apk vendor/asus/zenfone8/proprietary/priv-app/AsusGallery/AsusGallery.apk: Zip archive data, at least v0.0 to extract, compression method=deflate [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

Building the image fails with an error like this: fatal error: 'sound/voice_params.h' file not found

ERROR: fatal error: 'sound/voice_params.h' file not found

If the build of the OmniROM installation image fails with an error message like this

hardware/qcom-caf/sm8350/audio/hal/msm8974/platform.h:26:10: fatal error: 'sound/voice_params.h' file not found #include <sound/voice_params.h>

or

hardware/qcom-caf/sm8350/audio/hal/audio_extn/spkr_protection.c:86:10: fatal error: 'linux/msm_audio_calibration.h' file not found #include <linux/msm_audio_calibration.h>
you may try this approach:

# init the environment # . build/envsetup.sh # cleanup the out directory (optional) # m clean # select the phone model # lunch omni_zenfone8-user # create the kernel # (this command will only create the kernel, e.g. out/target/product/zenfone8/kernel) # mka kernel # now create the complete image # brunch omni_zenfone8-user

If building the OS image fails again after a executing "repo sync", recreate the kernel again using these commands
lunch omni_zenfone8-user mka kernel

before creating the OS image

Building the image fails with an error like this: soong bootstrap failed with: exit status 1

ERROR: FAILED: out/soong/build.ninja

If creating the CustomROM image fails with an error message like this

[100% 3/3] analyzing Android.bp files and generating ninja file at out/soong/build.ninja FAILED: out/soong/build.ninja cd "$(dirname "out/host/linux-x86/bin/soong_build")" && BUILDER="$PWD/$(basename "out/host/linux-x86/bin/soong_build")" && cd / && env -i "$BUILDER" --to p "$TOP" --soong_out "out/soong" --out "out" -o out/soong/build.ninja --bazel-mode --globListDir build --globFile out/soong/globs-build.ninja -t - l out/.module_paths/Android.bp.list --available_env out/soong/soong.environment.available --used_env out/soong/soong.environment.used.build Android.bp 22:49:28 soong bootstrap failed with: exit status 1
and no other messages it might be because of an out of memory error. In this case check the OS log file for OOM messages, e.g.:

xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$ sudo grep -i "Out of memory" /var/log/messages | tail -2 Feb 7 22:49:25 fedora01 kernel: Out of memory: Killed process 358524 (soong_build) total-vm:39409176kB, anon-rss:25785600kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:64432kB oom_score_adj:0 Feb 7 23:30:59 fedora01 kernel: Out of memory: Killed process 361853 (soong_build) total-vm:40314564kB, anon-rss:25975352kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:65144kB oom_score_adj:0 xtrnaw7@fedora01:/data/develop/android/OmniROM_14.0$

-> soong_build is one of the processes used to build the image

To fix it either add more memory or swap. Another possible workaround for virtual machine is to remove CPUs from the virtual machine; the build process creates parallel running tasks depending on the number of CPUs and each process needs it own memory workload (according to aunt google you should calculate about 4 GB memory per used CPU plus 2 GB for the general tasks for the build)

To test if the command will work with less CPUs try these commands in the root directory for the repository:

. /build/envsetup m -j4

(where 4 is the number of CPUs that should be used for the compile)

In Linux you might also use the command taskset to start a process with restricted CPU usage, e.g.:

taskset -c 0-2 bash

will start a bash that can only use the CPUs 0,1, and 2. This is also true for all processes started from within this shell

Building the image fails with an error like this: libcrypto's ABI has EXTENDING CHANGES. Please check compatibility report

ERROR: libcrypto's ABI has EXTENDING CHANGES. Please check compatibility report

If creating the CustomROM image fails with an error message like this:

[ 81% 144559/176309] //external/boringssl:libcrypto header-abi-diff libcrypto.so.abidiff FAILED: out/soong/.intermediates/external/boringssl/libcrypto/android_vendor.34_arm64_armv8-a_shared/libcrypto.so.abidiff (prebuilts/clang-tools/linux-x86/bin/header-abi-diff -target-version current -allow-unreferenced-changes -allow-unreferenced-elf-symbol-changes -lib libcrypto -arc h arm64 -o out/soong/.intermediates/external/boringssl/libcrypto/android_vendor.34_arm64_armv8-a_shared/libcrypto.so.abidiff -new out/soong/.intermediates/external /boringssl/libcrypto/android_vendor.34_arm64_armv8-a_shared/libcrypto.so.lsdump -old prebuilts/abi-dumps/vndk/34/64/arm64/source-based/libcrypto.so.lsdump)|| (echo 'error: Please update ABI references with: $ANDROID_BUILD_TOP/development/vndk/tools/header-checker/utils/create_reference_dumps.py -l libcrypto' && (mkdir -p $DI ST_DIR/abidiffs && cp out/soong/.intermediates/external/boringssl/libcrypto/android_vendor.34_arm64_armv8-a_shared/libcrypto.so.abidiff $DIST_DIR/abidiffs/) && exi t 1) ****************************************************** error: libcrypto's ABI has EXTENDING CHANGES. Please check compatibility report at: out/soong/.intermediates/external/boringssl/libcrypto/android_vendor.34_arm64_a rmv8-a_shared/libcrypto.so.abidiff ****************************************************** error: Please update ABI references with: $ANDROID_BUILD_TOP/development/vndk/tools/header-checker/utils/create_reference_dumps.py -l libcrypto 22:11:58 ninja failed with: exit status 1 #### failed to build some targets (04:21:04 (hh:mm:ss)) ####

fix the error using these commands:

. ./build/envsetup.sh lunch omni_zenfone8-user ${ANDROID_BUILD_TOP}/development/vndk/tools/header-checker/utils/create_reference_dumps.py -l libcrypto -products $TARGET_PRODUCT --build-variant $TARGET_BUILD_VARIANT

see also this post: https://xdaforums.com/t/major-bug-in-android-14-breaks-building-aosp-on-debian-ubuntu.4636394/

AFAIK this error may occurs after the openssl libraries of the OS used to compile the Android image are updated to a new version.

Building the image fails with error messages about duplicat modules

ERROR: module "qcom.fmradio_defaults" already defined

If creating the CustomROM image fails with an error message like this:

error: vendor/qcom/opensource/fm-commonsys/qcom/fmradio/Android.bp:1:1: module "qcom.fmradio_defaults" already defined vendor/qcom/opensource/commonsys/fm/qcom/fmradio/Android.bp:1:1 <-- previous definition here error: vendor/qcom/opensource/fm-commonsys/qcom/fmradio/Android.bp:6:1: module "qcom.fmradio" already defined vendor/qcom/opensource/commonsys/fm/qcom/fmradio/Android.bp:6:1 <-- previous definition here error: vendor/qcom/opensource/fm-commonsys/qcom/fmradio/Android.bp:14:1: module "qcom.fmradio.xml" already defined vendor/qcom/opensource/commonsys/fm/qcom/fmradio/Android.bp:14:1 <-- previous definition here ninja: build stopped: subcommand failed.
remove the roomservice

rm .repo/local_manifests/roomservice.xml

and try again

Executing lunch omni_zenfone8-user fails with "Invalid lunch combo"

ERROR: lunch fails with "Invalid lunch combo"

If the lunch, brunch, or breakfast command fails with an error message like this

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ lunch omni_zenfone8-user Invalid lunch combo: omni_zenfone8-user Valid combos must be of the form <product>-<release>-<variant> [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $this is most probably caused by the new version configuration in AOSP since Q1 2024 -- see Infos for building a CustomROM image for details

Building the image fails with the error thread exhaustion

ERROR: program exceeds 10000-thread limit

If building the image fails because a thread limit was exceeded:

runtime: program exceeds 10000-thread limit fatal error: thread exhaustion
this may be caused by invalid symbolic links in the directory ./out and you moved or copied the build tree to a new directory.

There are a lot of absolute symbolic links in the directory ./out in the build tree that do not work anymore after moving or copying the directory tree.
To fix it either correct the invalid symbolic links or clean the complete ./out directory.

History of this entry

08.02.2024

initial release

09.02.2024

added hint about missing environment setup
added hint about how fix libcrypto AIB error

02.03.2024

added hint about fixing duplicate module definitions

29.03.2024

added the hint about Executing lunch omni_zenfone8-user fails with "Invalid lunch combo"

24.06.2024

added the hint about Building the image fails with the error thread exhaustion

---

How to compile a C program for Android

URL: not yet published, but the entry is referenced in this post: https://xdaforums.com/t/how-to-compile-a-c-program-for-android-using-the-android-ndk-cli-tools.4656126/

How to compile a C program for Android using CLANG

Android is in the end just another Linux and therefore many C programs that were developed for Linux can also be compiled to run in the Android OS.

The tool chain required to compile a C program for Android is part of the Android NDK; the Android NDK can be downloaded from here:https://developer.android.com/ndk/downloads

The compiler in the current Android NDK is clang; to compile a program for Android using gcc one of the old Android SDK versions with gcc is necessary -- see How to compile a C program for Android using gcc

Android NDKs are available as tar file -- to install the Android NDK just download the tar file with the Android NDK and untar it.

The example below is for the Android NDK version r25.2 "installed"in the directory /data/develop/android/android-ndk-r25.2 on a machine running the Linux OS.

(NDK=/data/develop/android/android-ndk-r25.2)

The tool chain for compiling a C program for Android using clang is in the directory

${NDK}/toolchains/llvm/prebuilt/linux-x86_64/bin

e.g.

ls toolchains/llvm/prebuilt/linux-x86_64/bin

xtrnaw7@fedora01:/data/develop/android/android-ndk-r25.2$ ls toolchains/llvm/prebuilt/linux-x86_64/bin aarch64-linux-android21-clang armv7a-linux-androideabi24-clang clang-offload-packager i686-linux-android31-clang llvm-debuginfod llvm-profdata UnicodeNameMappingGenerator aarch64-linux-android21-clang++ armv7a-linux-androideabi24-clang++ clang-offload-wrapper i686-linux-android31-clang++ llvm-debuginfod-find llvm-profgen verify-uselistorder aarch64-linux-android22-clang armv7a-linux-androideabi26-clang clang-refactor i686-linux-android32-clang llvm-diff llvm-ranlib x86_64-linux-android21-clang aarch64-linux-android22-clang++ armv7a-linux-androideabi26-clang++ clang-rename i686-linux-android32-clang++ llvm-dis llvm-rc x86_64-linux-android21-clang++ aarch64-linux-android23-clang armv7a-linux-androideabi27-clang clang-repl i686-linux-android33-clang llvm-dlltool llvm-readelf x86_64-linux-android22-clang aarch64-linux-android23-clang++ armv7a-linux-androideabi27-clang++ clang-scan-deps i686-linux-android33-clang++ llvm-dwarfdump llvm-readobj x86_64-linux-android22-clang++ aarch64-linux-android24-clang armv7a-linux-androideabi28-clang clang-tidy intercept-build llvm-dwarfutil llvm-reduce x86_64-linux-android23-clang aarch64-linux-android24-clang++ armv7a-linux-androideabi28-clang++ count ld llvm-dwp llvm-remark-size-diff x86_64-linux-android23-clang++ aarch64-linux-android26-clang armv7a-linux-androideabi29-clang diagtool ld64.lld llvm-exegesis llvm-rtdyld x86_64-linux-android24-clang aarch64-linux-android26-clang++ armv7a-linux-androideabi29-clang++ dsymutil ld.lld llvm-extract llvm-sim x86_64-linux-android24-clang++ aarch64-linux-android27-clang armv7a-linux-androideabi30-clang FileCheck llc llvm-gsymutil llvm-size x86_64-linux-android26-clang aarch64-linux-android27-clang++ armv7a-linux-androideabi30-clang++ git-clang-format lld llvm-ifs llvm-split x86_64-linux-android26-clang++ aarch64-linux-android28-clang armv7a-linux-androideabi31-clang hmaptool lldb llvm-install-name-tool llvm-stress x86_64-linux-android27-clang aarch64-linux-android28-clang++ armv7a-linux-androideabi31-clang++ i686-linux-android19-clang lldb-argdumper llvm-jitlink llvm-strings x86_64-linux-android27-clang++ aarch64-linux-android29-clang armv7a-linux-androideabi32-clang i686-linux-android19-clang++ lldb.sh llvm-jitlink-executor llvm-strip x86_64-linux-android28-clang aarch64-linux-android29-clang++ armv7a-linux-androideabi32-clang++ i686-linux-android21-clang lld-link llvm-lib llvm-symbolizer x86_64-linux-android28-clang++ aarch64-linux-android30-clang armv7a-linux-androideabi33-clang i686-linux-android21-clang++ lli llvm-libtool-darwin llvm-tapi-diff x86_64-linux-android29-clang aarch64-linux-android30-clang++ armv7a-linux-androideabi33-clang++ i686-linux-android22-clang lli-child-target llvm-link llvm-tblgen x86_64-linux-android29-clang++ aarch64-linux-android31-clang bisect_driver.py i686-linux-android22-clang++ llvm-addr2line llvm-lipo llvm-tli-checker x86_64-linux-android30-clang aarch64-linux-android31-clang++ bugpoint i686-linux-android23-clang llvm-ar llvm-lto llvm-undname x86_64-linux-android30-clang++ aarch64-linux-android32-clang c-index-test i686-linux-android23-clang++ llvm-as llvm-lto2 llvm-windres x86_64-linux-android31-clang aarch64-linux-android32-clang++ clang i686-linux-android24-clang llvm-bcanalyzer llvm-mc llvm-xray x86_64-linux-android31-clang++ aarch64-linux-android33-clang clang++ i686-linux-android24-clang++ llvm-bitcode-strip llvm-mca merge-fdata x86_64-linux-android32-clang aarch64-linux-android33-clang++ clang-15 i686-linux-android26-clang llvm-bolt llvm-ml not x86_64-linux-android32-clang++ analyze-build clang-check i686-linux-android26-clang++ llvm-cat llvm-modextract obj2yaml x86_64-linux-android33-clang armv7a-linux-androideabi19-clang clang-cl i686-linux-android27-clang llvm-cfi-verify llvm-mt opt x86_64-linux-android33-clang++ armv7a-linux-androideabi19-clang++ clang-cpp i686-linux-android27-clang++ llvm-config llvm-nm remote_toolchain_inputs yaml2obj armv7a-linux-androideabi21-clang clangd i686-linux-android28-clang llvm-cov llvm-objcopy sancov yaml-bench armv7a-linux-androideabi21-clang++ clang-extdef-mapping i686-linux-android28-clang++ llvm-c-test llvm-objdump sanstats yasm armv7a-linux-androideabi22-clang clang-format i686-linux-android29-clang llvm-cvtres llvm-opt-report scan-build armv7a-linux-androideabi22-clang++ clang-linker-wrapper i686-linux-android29-clang++ llvm-cxxdump llvm-otool scan-build-py armv7a-linux-androideabi23-clang clang-nvlink-wrapper i686-linux-android30-clang llvm-cxxfilt llvm-pdbutil scan-view armv7a-linux-androideabi23-clang++ clang-offload-bundler i686-linux-android30-clang++ llvm-cxxmap llvm-PerfectShuffle split-file xtrnaw7@fedora01:/data/develop/android/android-ndk-r25.2$

The sysroot directory with the most frequently used include files and libraries required for compiling C programs is located in the directory

${NDK}/toolchains/llvm/prebuilt/linux-x86_64/sysroot

(see https://developer.android.com/ndk/guides/stable_apis for details)

The necessary changes to the Makefile for a C program to create an executable file for execution in the Android operating system using clang are:

1. change the compiler to be used to the appropriate C compiler of the Android NDK (in many Makefiles this is the variable CC)

CC=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android33-clang

2. Add the options

--sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE

to the options for the compiler (CFLAGS)

3. Add the options

--sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -pie

to the options for the linker (LDFLAGS)

4. Remove not necessary libraries from the makefile:

For example the library pthreads is not necessary for Android executables (the functions from pthreads are part of the bionic library in the Android OS).
(see https://developer.android.com/ndk/guides/stable_apis for details)

Note:

If the script ./configure is used to create the make files it's sufficient to define these environment variables before executing ./configure, e.g.:
# for the ASUS Zenfone 8 use # export TARGET=aarch64-linux-android # Set this to your minSdkVersion. export API=21 export NDK=/data/develop/android/android-ndk-r25.2 export TOOLCHAIN=${NDK}/toolchains/llvm/prebuilt/linux-x86_64 export CFLAGS="--sysroot ${TOOLCHAIN}/sysroot -fPIE" export LDFLAGS="--sysroot ${TOOLCHAIN}/sysroot -pie" export CC=$TOOLCHAIN/bin/$TARGET$API-clang export AR=${TOOLCHAIN}/bin/llvm-ar export AS=${CC} export CXX=${TOOLCHAIN}/bin/$TARGET$API-clang++ export LD=${TOOLCHAIN}/bin/ld export RANLIB=${TOOLCHAIN}/bin/llvm-ranlib export STRIP=${TOOLCHAIN}/bin/llvm-strip ./configure --host ${TARGET}

(see also https://developer.android.com/ndk/guides/other_build_systems)

Example 1 - create the executable "pigz" for Android

Compile the C program pigz (http://zlib.net/pigz/) with the compiler clang into an executable file for the Android operating system.

pigz can be compiled using the include files and libraries in the sysroot directory from the Android NDK (no other libraries are required)

Compile the executable pigz

1. Download the source code for pigz using

wget http://zlib.net/pigz/pigz-2.8.tar.gz

2. Unpack the tar file
tar -xf pigz-2.8.tar.gz
cd pigz-2.8

3. Correct the make file for pigz

cp Makefile Makefile.org

# edit the Makefile ; the differences when done should look like this:

xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$ diff Makefile Makefile.org 1,3c1,3 < CC=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang < CFLAGS= --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual < LDFLAGS=--sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -pie --- > CC=gcc > CFLAGS=-O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual > LDFLAGS= 8,9c8 < #LIBS=-lm -lpthread -lz < LIBS=-lm -lz --- > LIBS=-lm -lpthread -lz xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$

4. Create the executable using make

make

Example output of the make command

xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$ make /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c -o pigz.o pigz.c /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c -o yarn.o yarn.c /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c -o try.o try.c /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/deflate.c /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/blocksplitter.c /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/tree.c /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/lz77.c /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/cache.c /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/hash.c /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/util.c /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/squeeze.c zopfli/src/zopfli/squeeze.c:346:10: warning: variable 'total_length_test' set but not used [-Wunused-but-set-variable] size_t total_length_test = 0; ^ 1 warning generated. /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/katajainen.c /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/symbols.c /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang --sysroot= --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -pie -o pigz pigz.o yarn.o try.o deflate.o blocksplitter.o tree.o lz77.o cache.o hash.o util.o squeeze.o katajainen.o symbols.o -lm -lz ln -f pigz unpigz xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$

5. Check the result

xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$ ls -l pigz -rwxr-xr-x 2 xtrnaw7 xtrnaw7 144256 Feb 12 07:20 pigz xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$ xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$ file pigz pigz: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /system/bin/linker64, not stripped xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$

6. Copy the executable pigz to the phone and test it

adb push pigz /sdcard/Download

And then in an adb shell on the phone:

ASUS_I006D:/ $ cp /sdcard/Download/pigz /data/local/tmp/pigz
ASUS_I006D:/ $

ASUS_I006D:/ $ chmod 755 /data/local/tmp/pigz
ASUS_I006D:/ $

ASUS_I006D:/ $ file /data/local/tmp/pigz
/data/local/tmp/pigz: ELF shared object, 64-bit LSB arm64, dynamic (/system/bin/linker64), for Android 21, built by NDK r25 (8775105), not stripped
ASUS_I006D:/ $

ASUS_I006D:/ $ /data/local/tmp/pigz -V
pigz 2.8
ASUS_I006D:/ $

For libraries required by an executable that are not part of the Android NDK I recommend to create an additional sysroot environment, e.g.:
mkdir -p /data/develop/android/android_root mkdir -p /data/develop/android/android_root/usr mkdir -p /data/develop/android/android_root/usr/bin mkdir -p /data/develop/android/android_root/usr/sbin mkdir -p /data/develop/android/android_root/usr/lib mkdir -p /data/develop/android/android_root/usr/include mkdir -p /data/develop/android/android_root/usr/man ln -s ./usr/bin /data/develop/android/android_root/bin ln -s ./usr/lib /data/develop/android/android_root/lib ln -s ./usr/sbin /data/develop/android/android_root/sbin

Then use the configure option --prefix=/data/develop/android/android_root/usr to compile and "install" the necessary libraries.

To use the include files and libaries from that sysroot environment to create executables for Android use the additional compiler option

CFLAGS=-I/data/develop/android/android_root/usr/include

and the additional linker option

LDFLAGS=-L/data/develop/android/android_root/usr/lib

Example 2 - create the "ncurses library" for Android

Compile the library ncurses, which is to be used in the Android OS, with the compiler clang.

Compile the library ncurses

Download and unpack the source files for ncurses:

wget https://ftp.gnu.org/gnu/ncurses/ncurses-6.3.tar.gz tar -xzf ncurses-6.3.tar.gz cd ncurses-6.3/

Prepare the environment for running ./configure to create the make files:

# for the ASUS Zenfone 8 use # export TARGET=aarch64-linux-android # Set this to your minSdkVersion. # export API=21 export NDK=/data/develop/android/android-ndk-r25.2 export TOOLCHAIN=${NDK}/toolchains/llvm/prebuilt/linux-x86_64 export CFLAGS="--sysroot ${TOOLCHAIN}/sysroot -fPIE" export LDFLAGS="--sysroot ${TOOLCHAIN}/sysroot -pie" export CC=${TOOLCHAIN}/bin/$TARGET$API-clang export AR=${TOOLCHAIN}/bin/llvm-ar export AS=${CC} export CXX=${TOOLCHAIN}/bin/$TARGET$API-clang++ export LD=${TOOLCHAIN}/bin/ld export RANLIB=${TOOLCHAIN}/bin/llvm-ranlib export STRIP=${TOOLCHAIN}/bin/llvm-strip

Use ./configure to create the make files:

exportSTRIPPROG=${STRIP} INSTALL=$PWD/install-sh ./configure --prefix=/data/develop/android/android_root/usr --host ${TARGET}

Notes:

At least in Fedora 39 the executable /usr/bin/install (which is used by default in the script configure) uses the binary strip found in the PATH and ignores the environment variable. But the strip from Fedora can not handle executables for the ARM CPUs so it can not be used for installing files for ARM CPUs. STRIPPROG Is the environment variable used in install-sh for the strip executable to use.

Important are the last lines of the output of ./configure:

** Configuration summary for NCURSES 6.3 20211021: extended funcs: yes xterm terminfo: xterm-new bin directory: /data/develop/android/android_root/usr/bin lib directory: /data/develop/android/android_root/usr/lib include directory: /data/develop/android/android_root/usr/include/ncurses man directory: /data/develop/android/android_root/usr/share/man terminfo directory: /data/develop/android/android_root/usr/share/terminfo ** Include-directory is not in a standard location

Now create the library using make:

make

When done install the ncurses library and include files in the sysroot directory for Android :

make install

To check the result list the files in the sysroot directory:

example

xtrnaw7@fedora01:/data/develop/android/source/ncurses-6.3$ ls -l /data/develop/android/android_root/usr/include/ total 2 drwxr-xr-x 2 xtrnaw7 xtrnaw7 11 Feb 12 13:32 ncurses lrwxrwxrwx 1 xtrnaw7 xtrnaw7 19 Feb 12 13:20 ncurses.h -> ./ncurses/ncurses.h xtrnaw7@fedora01:/data/develop/android/source/ncurses-6.3$ xtrnaw7@fedora01:/data/develop/android/source/ncurses-6.3$ ls -l /data/develop/android/android_root/usr/lib/ total 1801 -rw-r--r-- 1 xtrnaw7 xtrnaw7 848880 Feb 12 13:32 libncurses.a -rw-r--r-- 1 xtrnaw7 xtrnaw7 3303892 Feb 12 13:32 libncurses_g.a lrwxrwxrwx 1 xtrnaw7 xtrnaw7 17 Feb 12 13:30 terminfo -> ../share/terminfo xtrnaw7@fedora01:/data/develop/android/source/ncurses-6.3$

Example 3 - create the executable "bvi" for Android; bvi requires the ncurses library

Compile the C program bvi into an executable for the Android OS using the clang.
bvi requires the library ncurses which is not part of the sysroot from the Android NDK.

Compile the executable bvi

Download the source for bvi

wget http://sourceforge.net/projects/bvi/files/bvi/1.4.2/bvi-1.4.2.src.tar.gz/download -O bvi-1.4.2.src.tar.gz

and unpack the tar file

tar -xf bvi-1.4.2.src.tar.gz cd bvi-1.4.2

Prepare the environment for running ./configure to create the make files:

# for the ASUS Zenfone 8 use # export TARGET=aarch64-linux-android # Set this to your minSdkVersion. # export API=21 export NDK=/data/develop/android/android-ndk-r25.2 export TOOLCHAIN=${NDK}/toolchains/llvm/prebuilt/linux-x86_64 export CFLAGS="--sysroot ${TOOLCHAIN}/sysroot -fPIE" export LDFLAGS="--sysroot ${TOOLCHAIN}/sysroot -pie" export CC=${TOOLCHAIN}/bin/$TARGET$API-clang export AR=${TOOLCHAIN}/bin/llvm-ar export AS=${CC} export CXX=${TOOLCHAIN}/bin/$TARGET$API-clang++ export LD=${TOOLCHAIN}/bin/ld export RANLIB=${TOOLCHAIN}/bin/llvm-ranlib export STRIP=${TOOLCHAIN}/bin/llvm-strip
# include the additional sysroot directories with the files for the ncurses library # export CFLAGS="-w --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -I/data/develop/android/android_root/usr/include -Wno-implicit-function-declaration -Wno-int-conversion" export LDFLAGS="--sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -pie -L/data/develop/android/android_root/usr/lib"

Note:

Normally you should fix the reason for the warnings and not suppress them with compiler options like -Wno-implicit-function-declaration -Wno-int-conversion.

Edit the input file config.h.in for the script configure to request the usage of the replacement function for rpl_malloc and rpl_realloc:
cp config.h.in config.h.in.org
# edit config.h.in and delete the lines: #undef realloc #undef malloc
->

xtrnaw7@fedora01:/data/develop/android/source/bvi/bvi-1.4.2$ diff config.h.in config.h.in.org 134a135 > #undef malloc 139a141 > #undef realloc xtrnaw7@fedora01:/data/develop/android/source/bvi/bvi-1.4.2$

Create the make files using the script ./configure:

./configure --prefix=/data/develop/android/android_root/usr --host $TARGET

Create the binary using make:

make

Check the result:

xtrnaw7@fedora01:/data/develop/android/source/bvi-1.4.2$ ls -l bvi -rwxr-xr-x 1 xtrnaw7 xtrnaw7 322840 Feb 12 11:48 bvi xtrnaw7@fedora01:/data/develop/android/source/bvi-1.4.2$ xtrnaw7@fedora01:/data/develop/android/source/bvi-1.4.2$ file bvi bvi: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /system/bin/linker64, not stripped xtrnaw7@fedora01:/data/develop/android/source/bvi-1.4.2$

copy the binary to the phone:

adb push bvi /sdcard/Download

and then in an adb shell on the phone:

cp /sdcard/Download/bvi /data/local/tmp/ chmod 755 /data/local/tmp/bvi /data/local/tmp/bvi-h

example:

ASUS_I006D:/ $ /data/local/tmp/bvi -h Usage: bvi [-R] [-c cmd | +cmd] [-f script] [-s skip] [-e end] [-n length] file ... file offset/size: 10k, 20m, 1g, 0x1000 hex, 0200 octal 1|ASUS_I006D:/ $

Notes:

I recommend using a virtual machine to compile C programs for Android to avoid accidentally destroying your running Linux operating system.

If executing the new binary in the Android OS fails with an error message like this:

ASUS_I006D:/ $ /data/local/tmp/pigz "/data/local/tmp/pigz": error: Android 5.0 and later only support position-independent executables (-fPIE). 1|ASUS_I006D:/ $

the executable was compiled without the option -fPIE and/or linked without the options -pie.

Use the command readelf to check the compile options for the binary - examples:

readelf output for executables compiled without the options:

ASUS_I006D:/ $ readelf -l /data/local/tmp/pigz | grep "file type" Elf file type is EXEC (Executable file) ASUS_I006D:/ $

readelf output for executables compiled with the options:

ASUS_I006D:/ $ readelf -l /data/local/tmp/pigz | grep "file type" Elf file type is DYN (Shared object file) ASUS_I006D:/ $

To check if a function is available in the libc from an Android NDK use the tool nm, e.g.: to check if the function thread_create is available in the libc use this command:

xtrnaw7@fedora01:~$ nm /data/develop/android/android-ndk-r25.2/toolchains/llvm.dir/sysroot/usr/lib/aarch64-linux-android/26/libc.so | grep -w T | grep thread_create 000000000001b538 T pthread_create xtrnaw7@fedora01:~$

see also https://android.googlesource.com/platform/bionic/+/master/docs/status.md

see also Examples for compiling C programs for Android using gcc or clang

see also Compiling C programs in Android

History of this entry

12.02.2024

initial release

How to compile a C program for Android using gcc

URL: not yet published, but the entry is referenced in this post: https://xdaforums.com/t/how-to-compile-a-c-program-for-android-using-the-android-ndk-cli-tools.4656126/

How to compile a C program for running in Android using gcc

Android is in the end just another Linux and therefore many C programs that were developed for Linux can also be compiled to run in the Android OS.

The tool chain required to compile a C program for Android is part of the Android NDK; the Android NDK can be downloaded from here:https://developer.android.com/ndk/downloads

The current version of the Android NDK only contains the compiler clang. In principle clang should be able to compile all C programs but some make files for existing C programs require the compiler gcc.

The gcc compiler was part of the Android NDK in some of the earlier versions of the NDK. Therefor to compile a C program with the compiler gcc one of the earlier versions of the Android NDK must be used.

Old Android NDK versions are available here: https://github.com/android/ndk/wiki/Unsupported-Downloads

Android NDKs are available as tar file -- to install the Android NDK just download the tar file with the Android NDK and untar it.

The example below is for the Android NDK version r14b "installed"in the directory /data/develop/android/android-ndk-r14b on a machine running the Linux OS.
(NDK=/data/develop/android/android-ndk-r14b)

The Android NDK version r14b contains an initial root directory with the include files and libraries most frequently used by Linux programs for various SDK versions in the directory

${NDK}/platforms

e.g.

ls -l platforms

xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$ ls -l platforms total 45 drwxr-xr-x 5 xtrnaw7 xtrnaw7 5 Mar 15 2017 android-12 drwxr-xr-x 5 xtrnaw7 xtrnaw7 5 Mar 15 2017 android-13 drwxr-xr-x 5 xtrnaw7 xtrnaw7 5 Mar 15 2017 android-14 drwxr-xr-x 5 xtrnaw7 xtrnaw7 5 Mar 15 2017 android-15 drwxr-xr-x 5 xtrnaw7 xtrnaw7 5 Mar 15 2017 android-16 drwxr-xr-x 5 xtrnaw7 xtrnaw7 5 Mar 15 2017 android-17 drwxr-xr-x 5 xtrnaw7 xtrnaw7 5 Mar 15 2017 android-18 drwxr-xr-x 5 xtrnaw7 xtrnaw7 5 Mar 15 2017 android-19 drwxr-xr-x 8 xtrnaw7 xtrnaw7 8 Mar 15 2017 android-21 drwxr-xr-x 8 xtrnaw7 xtrnaw7 8 Mar 15 2017 android-22 drwxr-xr-x 8 xtrnaw7 xtrnaw7 8 Mar 15 2017 android-23 drwxr-xr-x 8 xtrnaw7 xtrnaw7 8 Mar 15 2017 android-24 drwxr-xr-x 5 xtrnaw7 xtrnaw7 5 Mar 15 2017 android-9 -rw-r--r-- 1 xtrnaw7 xtrnaw7 312566 Mar 15 2017 NOTICE -rw-r--r-- 1 xtrnaw7 xtrnaw7 4465 Mar 15 2017 repo.prop xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$

In each sub directory for the SDK versions are directories for the CPUs used in Android devices, e.g:

xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$ ls -l platforms/android-24/ total 3 drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Mar 15 2017 arch-arm drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Mar 15 2017 arch-arm64 drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Mar 15 2017 arch-mips drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Mar 15 2017 arch-mips64 drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Mar 15 2017 arch-x86 drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Mar 15 2017 arch-x86_64 xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$

In most cases it's the best choice to use the newest sysroot directory tree for the CPU used in your phone, e.g. for the ASUS Zenfone 8 this is ${NDK}/platforms/android-24/arch-arm64 e.g.xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$ ls -l platforms/android-24/arch-arm64/* total 6 drwxr-xr-x 22 xtrnaw7 xtrnaw7 91 Mar 15 2017 include drwxr-xr-x 2 xtrnaw7 xtrnaw7 28 Mar 15 2017 lib xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$

(see https://developer.android.com/ndk/guides/stable_apis for details)

The compiler and tools to compile the C programs are in sub directories in the directory

${NDK}/toolchains

e.g.:

ls -l toolchains/

xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$ ls -l toolchains/ total 4 drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Mar 15 2017 aarch64-linux-android-4.9 drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Mar 15 2017 arm-linux-androideabi-4.9 drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Jun 26 2022 llvm drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Mar 15 2017 mips64el-linux-android-4.9 drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Mar 15 2017 mipsel-linux-android-4.9 drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Mar 15 2017 renderscript drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Jun 26 2022 x86-4.9 drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Mar 15 2017 x86_64-4.9 xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$

To compile the C program on a machine running Linux use the executables from the directory

${NDK}/toolchains/x86_64-4.9/prebuilt/linux-x86_64/bin

e.g.:

ls -l /data/develop/android/android-ndk-r14b/toolchains/x86_64-4.9/prebuilt/linux-x86_64/bin

xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$ ls -l /data/develop/android/android-ndk-r14b/toolchains/x86_64-4.9/prebuilt/linux-x86_64/bin total 18062 -rwxr-xr-x 1 xtrnaw7 xtrnaw7 937592 Mar 15 2017 x86_64-linux-android-addr2line -rwxr-xr-x 1 xtrnaw7 xtrnaw7 966176 Mar 15 2017 x86_64-linux-android-ar -rwxr-xr-x 1 xtrnaw7 xtrnaw7 1734464 Mar 15 2017 x86_64-linux-android-as -rwxr-xr-x 1 xtrnaw7 xtrnaw7 813576 Mar 15 2017 x86_64-linux-android-c++ -rwxr-xr-x 1 xtrnaw7 xtrnaw7 933080 Mar 15 2017 x86_64-linux-android-c++filt -rwxr-xr-x 1 xtrnaw7 xtrnaw7 813576 Mar 15 2017 x86_64-linux-android-cpp -rwxr-xr-x 1 xtrnaw7 xtrnaw7 2790568 Mar 15 2017 x86_64-linux-android-dwp -rwxr-xr-x 1 xtrnaw7 xtrnaw7 27976 Mar 15 2017 x86_64-linux-android-elfedit -rwxr-xr-x 1 xtrnaw7 xtrnaw7 813576 Mar 15 2017 x86_64-linux-android-g++ -rwxr-xr-x 1 xtrnaw7 xtrnaw7 809480 Mar 15 2017 x86_64-linux-android-gcc -rwxr-xr-x 1 xtrnaw7 xtrnaw7 809480 Mar 15 2017 x86_64-linux-android-gcc-4.9 -rwxr-xr-x 1 xtrnaw7 xtrnaw7 809480 Mar 15 2017 x86_64-linux-android-gcc-4.9.x -rwxr-xr-x 1 xtrnaw7 xtrnaw7 25440 Mar 15 2017 x86_64-linux-android-gcc-ar -rwxr-xr-x 1 xtrnaw7 xtrnaw7 25408 Mar 15 2017 x86_64-linux-android-gcc-nm -rwxr-xr-x 1 xtrnaw7 xtrnaw7 25408 Mar 15 2017 x86_64-linux-android-gcc-ranlib -rwxr-xr-x 1 xtrnaw7 xtrnaw7 426056 Mar 15 2017 x86_64-linux-android-gcov -rwxr-xr-x 1 xtrnaw7 xtrnaw7 462984 Mar 15 2017 x86_64-linux-android-gcov-tool -rwxr-xr-x 1 xtrnaw7 xtrnaw7 1000280 Mar 15 2017 x86_64-linux-android-gprof -rwxr-xr-x 1 xtrnaw7 xtrnaw7 4687880 Mar 15 2017 x86_64-linux-android-ld -rwxr-xr-x 1 xtrnaw7 xtrnaw7 1756288 Mar 15 2017 x86_64-linux-android-ld.bfd -rwxr-xr-x 1 xtrnaw7 xtrnaw7 4687880 Mar 15 2017 x86_64-linux-android-ld.gold -rwxr-xr-x 1 xtrnaw7 xtrnaw7 945432 Mar 15 2017 x86_64-linux-android-nm -rwxr-xr-x 1 xtrnaw7 xtrnaw7 1121336 Mar 15 2017 x86_64-linux-android-objcopy -rwxr-xr-x 1 xtrnaw7 xtrnaw7 2063256 Mar 15 2017 x86_64-linux-android-objdump -rwxr-xr-x 1 xtrnaw7 xtrnaw7 966176 Mar 15 2017 x86_64-linux-android-ranlib -rwxr-xr-x 1 xtrnaw7 xtrnaw7 441976 Mar 15 2017 x86_64-linux-android-readelf -rwxr-xr-x 1 xtrnaw7 xtrnaw7 937432 Mar 15 2017 x86_64-linux-android-size -rwxr-xr-x 1 xtrnaw7 xtrnaw7 937560 Mar 15 2017 x86_64-linux-android-strings -rwxr-xr-x 1 xtrnaw7 xtrnaw7 1121368 Mar 15 2017 x86_64-linux-android-strip xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$

The necessary changes to the Makefile for a C program to create an executable file for execution in the Android operating system are:

1. change the compiler to use to the approbiate C Compiler from the Android NDK (in a lot of Makefiles this is the variable CC)

CC=/data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9

2. Add the options

--sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -fPIE

to the options for the compiler (CFLAGS)

3. Add the options

--sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -pie

to the options for the linker (LDFLAGS)

4. Remove not necessary libraries from the makefile:

For example the library pthreads is not necessary for Android executables (the functions from pthreads are part of the bionic library in the Android OS)
(see https://developer.android.com/ndk/guides/stable_apis for details)

Note:

If the script ./configure is used to create the make files it's sufficient to define these enviroment variables before executing ./configure:

export NDK="/data/develop/android/android-ndk-r14b" # Set this to your minSdkVersion. # export API=21 # for the ASUS Zenfone 8 use # export TARGET=aarch64-linux-android export CFLAGS="--sysroot=${NDK}/platforms/android-${API}/arch-arm64 -fPIE" export LDFLAGS="--sysroot=${NDK}/platforms/android-${API}/arch-arm64 -pie" export CC=${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-gcc-4.9 export AR=${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-gcc-ar export LD=${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-gcc-ld export CXX=${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-c++ export RANLIB=${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-ranlib export STRIP=${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-strip export AS=${CC} ./configure --host ${TARGET}

Example 1 - create the executable "pigz" for Android

Compile the C program pigz (http://zlib.net/pigz/) with the compiler gcc into an executable file for the Android operating system.

pigz can be compiled using the include files and libraries in the sysroot directory from the Android NDK (no other libraries are required)

Compile the executable pigz

1. Download the source code for pigz using

wget http://zlib.net/pigz/pigz-2.8.tar.gz

2. Unpack the tar file
tar -xf pigz-2.8.tar.gz
cd pigz-2.8

3. Correct the make file for pigz

cp Makefile Makefile.org

# edit the Makefile ; the differences when done should look like this:

xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$ diff Makefile Makefile.org 1,3c1,3 < CC=/data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 < CFLAGS=--sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -fPIE-O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual < LDFLAGS=--sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -pie --- > CC=gcc > CFLAGS=-O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual > LDFLAGS= 8,9c8 < #LIBS=-lm -lpthread -lz <LIBS=-lm -lz --- > LIBS=-lm -lpthread -lz xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$

4. Create the executable using make

make

Example output of the make command

xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$ make /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c -o pigz.o pigz.c /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c -o yarn.o yarn.c /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c -o try.o try.c /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/deflate.c /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/blocksplitter.c /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/tree.c /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/lz77.c /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/cache.c /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/hash.c /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/util.c /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/squeeze.c /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/katajainen.c /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -fPIE -O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual -c zopfli/src/zopfli/symbols.c /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 -pie -o pigz pigz.o yarn.o try.o deflate.o blocksplitter.o tree.o lz77.o cache.o hash.o util.o squeeze.o katajainen.o symbols.o -lm -lz ln -f pigz unpigz xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$

5. Check the result

xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$ ls -l pigz -rwxr-xr-x 2 xtrnaw7 xtrnaw7 148688 Feb 11 21:03 pigz xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$ file pigz pigz: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /system/bin/linker64, not stripped xtrnaw7@fedora01:/data/develop/android/source/pigz-2.8$

6. Copy the executable pigz to the phone and test it

adb push pigz /sdcard/Download

And then in an adb shell on the phone:

ASUS_I006D:/ $ mv /sdcard/Download/pigz /data/local/tmp/ ASUS_I006D:/ $ ASUS_I006D:/ $ chmod 755 /data/local/tmp/pigz ASUS_I006D:/ $ ASUS_I006D:/ $ /data/local/tmp/pigz -V pigz 2.8 ASUS_I006D:/ $

For libraries required by an executable that are not part of the Android NDK I recommend to create an additional sysroot environment, e.g.:

mkdir -p /data/develop/android/android_root mkdir -p /data/develop/android/android_root/usr mkdir -p /data/develop/android/android_root/usr/bin mkdir -p /data/develop/android/android_root/usr/sbin mkdir -p /data/develop/android/android_root/usr/lib mkdir -p /data/develop/android/android_root/usr/include mkdir -p /data/develop/android/android_root/usr/man ln -s ./usr/bin /data/develop/android/android_root/bin ln -s ./usr/lib /data/develop/android/android_root/lib ln -s ./usr/sbin /data/develop/android/android_root/sbin

Then use the configure option --prefix=/data/develop/android/android_root/usr to compile and "install" necessary libraries.

To use the include files and libaries from that sysroot environment use the additional compiler option

CFLAGS=-I/data/develop/android/android_root/usr/include

and the additional linker option

LDFLAGS=-L/data/develop/android/android_root/usr/lib

Example 2 - create the "ncurses library" for Android

Compile the library ncurses, which is to be used in the Android OS, with the compiler gcc.

Compile the library ncurses

Download and unpack the source files for ncurses:
wget https://ftp.gnu.org/gnu/ncurses/ncurses-6.3.tar.gz tar -xzf ncurses-6.3.tar.gz cd ncurses-6.3/

Prepare the environment for running ./configure to create the make files:

# for the ASUS Zenfone 8 use # export TARGET=aarch64-linux-android # Set this to your minSdkVersion. # export API=21 export NDK="/data/develop/android/android-ndk-r14b" export CFLAGS="--sysroot=${NDK}/platforms/android-${API}/arch-arm64 -fPIE" export LDFLAGS="--sysroot=${NDK}/platforms/android-${API}/arch-arm64 -pie" export CC="${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-gcc-4.9" export AR="${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-gcc-ar" export LD="${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-gcc-ld" export CXX="${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-c++" export RANLIB="${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-ranlib" export STRIP="${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-strip" export AS="${CC}"

Use ./configure to create the make files:

export STRIPPROG=${STRIP} INSTALL=$PWD/install-sh ./configure --prefix=/data/develop/android/android_root/usr --host ${TARGET}

Notes:

At least in Fedora 39 the executable /usr/bin/install (which is used by default in the script ./configure) uses the binary strip found in the PATH and ignores the environment variable. But the strip from Fedora can not handle executables for the ARM CPUs so it can not be used for installing files for ARM CPUs. STRIPPROG Is the environment variable used in install-sh for the strip executable to use.

Important are the last lines of the output of ./configure:

** Configuration summary for NCURSES 6.3 20211021: extended funcs: yes xterm terminfo: xterm-new bin directory: /data/develop/android/android_root/usr/bin lib directory: /data/develop/android/android_root/usr/lib include directory: /data/develop/android/android_root/usr/include/ncurses man directory: /data/develop/android/android_root/usr/share/man terminfo directory: /data/develop/android/android_root/usr/share/terminfo ** Include-directory is not in a standard location

Now create the library using make:

make

When done install the ncurses library and include files in the sysroot directory for Android :

make install

To check the result list the files in the sysroot directory:

example

xtrnaw7@fedora01:/data/develop/android/source/bvi-1.4.2$ ls -l /data/develop/android/android_root/usr/include/ total 2 drwxr-xr-x 2 xtrnaw7 xtrnaw7 11 Feb 12 15:27 ncurses lrwxrwxrwx 1 xtrnaw7 xtrnaw7 19 Feb 12 13:20 ncurses.h -> ./ncurses/ncurses.h xtrnaw7@fedora01:/data/develop/android/source/bvi-1.4.2$ xtrnaw7@fedora01:/data/develop/android/source/bvi-1.4.2$ ls -l /data/develop/android/android_root/usr/lib total 1770 -rw-r--r-- 1 xtrnaw7 xtrnaw7 808984 Feb 12 15:27 libncurses.a -rw-r--r-- 1 xtrnaw7 xtrnaw7 3571844 Feb 12 15:27 libncurses_g.a lrwxrwxrwx 1 xtrnaw7 xtrnaw7 17 Feb 12 13:30 terminfo -> ../share/terminfo xtrnaw7@fedora01:/data/develop/android/source/bvi-1.4.2$

Example 3 - create the executable "bvi" for Android; bvi requires the ncurses library

Compile the C program bvi into an executable for the Android OS using the gcc.
bvi requires the library ncurses which is not part of the sysroot from the Android NDK

(see the Examples for compiling C programs for Android using gcc or clang below for how to create a static linked bvi executable)

Compile the executable bvi

Download the source for bvi

wget http://sourceforge.net/projects/bvi/files/bvi/1.4.2/bvi-1.4.2.src.tar.gz/download -O bvi-1.4.2.src.tar.gz

and unpack the tar file

tar -xf bvi-1.4.2.src.tar.gz cd bvi-1.4.2

Prepare the environment for running ./configure to create the make files:

# for the ASUS Zenfone 8 use # export TARGET=aarch64-linux-android # Set this to your minSdkVersion. # export API=21 export NDK=/data/develop/android/android-ndk-r25.2 export TOOLCHAIN=${NDK}/toolchains/llvm/prebuilt/linux-x86_64 export CFLAGS="--sysroot ${TOOLCHAIN}/sysroot -fPIE" export LDFLAGS="--sysroot ${TOOLCHAIN}/sysroot -pie" export CC="${TOOLCHAIN}/bin/$TARGET$API-clang" export AR="${TOOLCHAIN}/bin/llvm-ar" export AS="${CC}" export CXX="${TOOLCHAIN}/bin/$TARGET$API-clang++" export LD="${TOOLCHAIN}/bin/ld" export RANLIB="${TOOLCHAIN}/bin/llvm-ranlib" export STRIP="${TOOLCHAIN}/bin/llvm-strip" # include the additional sysroot directories with the files for the ncurses library # export CFLAGS="-w --sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIE -I/data/develop/android/android_root/usr/include -Wno-implicit-function-declaration -Wno-int-conversion" export LDFLAGS="--sysroot=/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot -pie -L/data/develop/android/android_root/usr/lib"

Note:

Normally you should fix the reason for the warnings and not suppress them with compiler options like -Wno-implicit-function-declaration -Wno-int-conversion.

Edit the input file config.h.in for the script ./configure to request the usage of the replacement function for rpl_malloc and rpl_realloc:
cp config.h.in config.h.in.org # edit config.h.in and delete the lines: #undef realloc #undef malloc

-> xtrnaw7@fedora01:/data/develop/android/source/bvi/bvi-1.4.2$ diff config.h.in config.h.in.org 134a135 > #undef malloc 139a141 > #undef realloc xtrnaw7@fedora01:/data/develop/android/source/bvi/bvi-1.4.2$

Create the make files using the script ./configure:

./configure --prefix=/data/develop/android/android_root/usr --host $TARGET

Create the binary using make:

make

Check the result:

xtrnaw7@fedora01:/data/develop/android/source/bvi-1.4.2$ ls -l bvi -rwxr-xr-x 1 xtrnaw7 xtrnaw7 322840 Feb 12 11:48 bvi xtrnaw7@fedora01:/data/develop/android/source/bvi-1.4.2$ xtrnaw7@fedora01:/data/develop/android/source/bvi-1.4.2$ file bvi bvi: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /system/bin/linker64, not stripped xtrnaw7@fedora01:/data/develop/android/source/bvi-1.4.2$

copy the binary to the phone:

adb push bvi /sdcard/Download

and then in an adb shell on the phone:

cp /sdcard/Download/bvi /data/local/tmp/ chmod 755 /data/local/tmp/bvi /data/local/tmp/bvi -h

example:

ASUS_I006D:/ $ /data/local/tmp/bvi -h Usage: bvi [-R] [-c cmd | +cmd] [-f script] [-s skip] [-e end] [-n length] file ... file offset/size: 10k, 20m, 1g, 0x1000 hex, 0200 octal 1|ASUS_I006D:/ $

In the Android NDK version r14b the script ./build/tools/make_standalone_toolchain.py can be used to create a tool chain directory for a specific environment, e.g:

./build/tools/make_standalone_toolchain.py --arch arm64 --install-dir /data/develop/android/rootdir_r14b

/build/tools/make_standalone_toolchain.py --arch arm64 --install-dir /data/develop/android/rootdir_r14b

xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$ ./build/tools/make_standalone_toolchain.py --arch arm64 --install-dir /data/develop/android/rootdir_r14b WARNING:__main__:Defaulting to target API 21 (minimum supported target for arm64) xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$ xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$ ls -l /data/develop/android/rootdir_r14b total 124 drwxr-xr-x 5 xtrnaw7 xtrnaw7 5 Feb 13 08:36 aarch64-linux-android -rw-r--r-- 1 xtrnaw7 xtrnaw7 11 Mar 15 2017 AndroidVersion.txt drwxr-xr-x 10 xtrnaw7 xtrnaw7 75 Feb 13 08:36 bin -rw-r--r-- 1 xtrnaw7 xtrnaw7 18002 Mar 15 2017 COPYING -rw-r--r-- 1 xtrnaw7 xtrnaw7 35147 Mar 15 2017 COPYING3 -rw-r--r-- 1 xtrnaw7 xtrnaw7 7639 Mar 15 2017 COPYING3.LIB -rw-r--r-- 1 xtrnaw7 xtrnaw7 26527 Mar 15 2017 COPYING.LIB -rw-r--r-- 1 xtrnaw7 xtrnaw7 3324 Mar 15 2017 COPYING.RUNTIME drwxr-xr-x 4 xtrnaw7 xtrnaw7 4 Feb 13 08:36 include drwxr-xr-x 7 xtrnaw7 xtrnaw7 8 Feb 13 08:36 lib drwxr-xr-x 3 xtrnaw7 xtrnaw7 6 Feb 13 08:36 lib64 drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Feb 13 08:36 libexec -rw-r--r-- 1 xtrnaw7 xtrnaw7 8516 Mar 15 2017 manifest_3362437.xml -rw-r--r-- 1 xtrnaw7 xtrnaw7 0 Mar 15 2017 MODULE_LICENSE_BSD_LIKE -rw-r--r-- 1 xtrnaw7 xtrnaw7 0 Mar 15 2017 MODULE_LICENSE_GPL -rw-r--r-- 1 xtrnaw7 xtrnaw7 0 Mar 15 2017 MODULE_LICENSE_MIT -rw-r--r-- 1 xtrnaw7 xtrnaw7 126349 Mar 15 2017 NOTICE -rw-r--r-- 1 xtrnaw7 xtrnaw7 4465 Mar 15 2017 repo.prop drwxr-xr-x 5 xtrnaw7 xtrnaw7 5 Feb 13 08:36 share drwxr-xr-x 3 xtrnaw7 xtrnaw7 3 Mar 15 2017 sysroot drwxr-xr-x 7 xtrnaw7 xtrnaw7 7 Feb 13 08:36 test drwxr-xr-x 4 xtrnaw7 xtrnaw7 4 Feb 13 08:36 tools xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$ xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$ ls -l /data/develop/android/rootdir_r14b/bin total 91048 -rwxr-xr-x 1 xtrnaw7 xtrnaw7 197 Mar 15 2017 2to3 -rwxr-xr-x 1 xtrnaw7 xtrnaw7 859832 Mar 15 2017 aarch64-linux-android-addr2line -rwxr-xr-x 1 xtrnaw7 xtrnaw7 888416 Mar 15 2017 aarch64-linux-android-ar -rwxr-xr-x 1 xtrnaw7 xtrnaw7 1336928 Mar 15 2017 aarch64-linux-android-as -rwxr-xr-x 1 xtrnaw7 xtrnaw7 776712 Mar 15 2017 aarch64-linux-android-c++ -rwxr-xr-x 1 xtrnaw7 xtrnaw7 855352 Mar 15 2017 aarch64-linux-android-c++filt -rwxr-xr-x 1 xtrnaw7 xtrnaw7 218 Feb 13 08:36 aarch64-linux-android-clang -rwxr-xr-x 1 xtrnaw7 xtrnaw7 222 Feb 13 08:36 aarch64-linux-android-clang++ -rwxr-xr-x 1 xtrnaw7 xtrnaw7 772968 Mar 15 2017 aarch64-linux-android-cpp -rwxr-xr-x 1 xtrnaw7 xtrnaw7 2790568 Mar 15 2017 aarch64-linux-android-dwp -rwxr-xr-x 1 xtrnaw7 xtrnaw7 27976 Mar 15 2017 aarch64-linux-android-elfedit -rwxr-xr-x 1 xtrnaw7 xtrnaw7 776712 Mar 15 2017 aarch64-linux-android-g++ -rwxr-xr-x 1 xtrnaw7 xtrnaw7 772616 Mar 15 2017 aarch64-linux-android-gcc -rwxr-xr-x 1 xtrnaw7 xtrnaw7 772616 Mar 15 2017 aarch64-linux-android-gcc-4.9 -rwxr-xr-x 1 xtrnaw7 xtrnaw7 772616 Mar 15 2017 aarch64-linux-android-gcc-4.9.x -rwxr-xr-x 1 xtrnaw7 xtrnaw7 25440 Mar 15 2017 aarch64-linux-android-gcc-ar -rwxr-xr-x 1 xtrnaw7 xtrnaw7 25408 Mar 15 2017 aarch64-linux-android-gcc-nm -rwxr-xr-x 1 xtrnaw7 xtrnaw7 25408 Mar 15 2017 aarch64-linux-android-gcc-ranlib -rwxr-xr-x 1 xtrnaw7 xtrnaw7 426056 Mar 15 2017 aarch64-linux-android-gcov -rwxr-xr-x 1 xtrnaw7 xtrnaw7 462984 Mar 15 2017 aarch64-linux-android-gcov-tool -rwxr-xr-x 1 xtrnaw7 xtrnaw7 922552 Mar 15 2017 aarch64-linux-android-gprof -rwxr-xr-x 1 xtrnaw7 xtrnaw7 1913056 Mar 15 2017 aarch64-linux-android-ld -rwxr-xr-x 1 xtrnaw7 xtrnaw7 1913056 Mar 15 2017 aarch64-linux-android-ld.bfd -rwxr-xr-x 1 xtrnaw7 xtrnaw7 4687880 Mar 15 2017 aarch64-linux-android-ld.gold -rwxr-xr-x 1 xtrnaw7 xtrnaw7 867896 Mar 15 2017 aarch64-linux-android-nm -rwxr-xr-x 1 xtrnaw7 xtrnaw7 1043608 Mar 15 2017 aarch64-linux-android-objcopy -rwxr-xr-x 1 xtrnaw7 xtrnaw7 1488280 Mar 15 2017 aarch64-linux-android-objdump -rwxr-xr-x 1 xtrnaw7 xtrnaw7 888416 Mar 15 2017 aarch64-linux-android-ranlib -rwxr-xr-x 1 xtrnaw7 xtrnaw7 441976 Mar 15 2017 aarch64-linux-android-readelf -rwxr-xr-x 1 xtrnaw7 xtrnaw7 859704 Mar 15 2017 aarch64-linux-android-size -rwxr-xr-x 1 xtrnaw7 xtrnaw7 859800 Mar 15 2017 aarch64-linux-android-strings -rwxr-xr-x 1 xtrnaw7 xtrnaw7 1043640 Mar 15 2017 aarch64-linux-android-strip drwxr-xr-x 2 xtrnaw7 xtrnaw7 4 Feb 13 08:36 arm64-v8a drwxr-xr-x 2 xtrnaw7 xtrnaw7 4 Feb 13 08:36 armeabi drwxr-xr-x 2 xtrnaw7 xtrnaw7 4 Feb 13 08:36 armeabi-v7a drwxr-xr-x 2 xtrnaw7 xtrnaw7 4 Feb 13 08:36 armeabi-v7a-hard -rwxr-xr-x 1 xtrnaw7 xtrnaw7 12628 Mar 15 2017 asan_device_setup -rwxr-xr-x 1 xtrnaw7 xtrnaw7 118848 Mar 15 2017 awk -rwxr-xr-x 1 xtrnaw7 xtrnaw7 218 Feb 13 08:36 clang -rwxr-xr-x 1 xtrnaw7 xtrnaw7 222 Feb 13 08:36 clang++ -rwxr-xr-x 1 xtrnaw7 xtrnaw7 41786976 Mar 15 2017 clang38 -rwxr-xr-x 1 xtrnaw7 xtrnaw7 41786976 Mar 15 2017 clang38++ -rwxr-xr-x 1 xtrnaw7 xtrnaw7 3459480 Mar 15 2017 clang-format -rwxr-xr-x 1 xtrnaw7 xtrnaw7 20736616 Mar 15 2017 clang-tidy -rwxr-xr-x 1 xtrnaw7 xtrnaw7 502144 Mar 15 2017 FileCheck -rwxr-xr-x 1 xtrnaw7 xtrnaw7 2945 Mar 15 2017 gcore -rwxr-xr-x 1 xtrnaw7 xtrnaw7 94 Mar 15 2017 gdb -rwxr-xr-x 1 xtrnaw7 xtrnaw7 9342192 Mar 15 2017 gdb-orig -rwxr-xr-x 1 xtrnaw7 xtrnaw7 195 Mar 15 2017 idle -rwxr-xr-x 1 xtrnaw7 xtrnaw7 2853264 Mar 15 2017 llvm-as -rwxr-xr-x 1 xtrnaw7 xtrnaw7 2267536 Mar 15 2017 llvm-dis -rwxr-xr-x 1 xtrnaw7 xtrnaw7 4733336 Mar 15 2017 llvm-link -rwxr-xr-x 1 xtrnaw7 xtrnaw7 3248048 Mar 15 2017 llvm-symbolizer -rwxr-xr-x 1 xtrnaw7 xtrnaw7 162752 Mar 15 2017 make drwxr-xr-x 2 xtrnaw7 xtrnaw7 4 Feb 13 08:36 mips drwxr-xr-x 2 xtrnaw7 xtrnaw7 4 Feb 13 08:36 mips64 -rwxr-xr-x 1 xtrnaw7 xtrnaw7 38368 Mar 15 2017 ndk-depends -rwxr-xr-x 1 xtrnaw7 xtrnaw7 87 Mar 15 2017 ndk-gdb -rwxr-xr-x 1 xtrnaw7 xtrnaw7 24982 Mar 15 2017 ndk-gdb.py -rwxr-xr-x 1 xtrnaw7 xtrnaw7 880576 Mar 15 2017 ndk-stack -rwxr-xr-x 1 xtrnaw7 xtrnaw7 1680 Mar 15 2017 ndk-which -rwxr-xr-x 1 xtrnaw7 xtrnaw7 180 Mar 15 2017 pydoc -rwxr-xr-x 1 xtrnaw7 xtrnaw7 2153751 Mar 15 2017 python -rwxr-xr-x 1 xtrnaw7 xtrnaw7 2153751 Mar 15 2017 python2 -rwxr-xr-x 1 xtrnaw7 xtrnaw7 2153751 Mar 15 2017 python2.7 -rwxr-xr-x 1 xtrnaw7 xtrnaw7 1770 Mar 15 2017 python2.7-config -rwxr-xr-x 1 xtrnaw7 xtrnaw7 1770 Mar 15 2017 python2-config -rwxr-xr-x 1 xtrnaw7 xtrnaw7 1770 Mar 15 2017 python-config -rwxr-xr-x 1 xtrnaw7 xtrnaw7 3161 Mar 15 2017 python-config.sh -rwxr-xr-x 1 xtrnaw7 xtrnaw7 18643 Mar 15 2017 smtpd.py drwxr-xr-x 2 xtrnaw7 xtrnaw7 4 Feb 13 08:36 x86 drwxr-xr-x 2 xtrnaw7 xtrnaw7 4 Feb 13 08:36 x86_64 -rwxr-xr-x 1 xtrnaw7 xtrnaw7 666280 Mar 15 2017 yasm xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$ xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$ ls -l /data/develop/android/rootdir_r14b/sysroot/ total 1 drwxr-xr-x 4 xtrnaw7 xtrnaw7 4 Mar 15 2017 usr xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$ xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$ ls -l /data/develop/android/rootdir_r14b/sysroot/usr total 6 drwxr-xr-x 20 xtrnaw7 xtrnaw7 89 Mar 15 2017 include drwxr-xr-x 2 xtrnaw7 xtrnaw7 26 Mar 15 2017 lib xtrnaw7@fedora01:/data/develop/android/android-ndk-r14b$

In this example, the compiler and the tools are located in the directory /data/develop/android/rootdir_r14b/bin and the sysroot is in the directory /data/develop/android/rootdir_r14b/sysroot.

Notes:

see also the notes for How to compile a C program for Android

The compiler in the current Android NDK is clang. See How to compile a C program for Android for how to compile a C program using clang from the current Android SDK.

see also Examples for compiling C programs for Android using gcc or clang

see also Compiling C programs in Android

History of this entry

12.02.2024

initial release

16.08.2024

corrected some typos in the export commands introduced while doing the page cleanup

Examples for compiling C programs for Android using gcc or clang

URL: not yet published, but the entry is referenced in this post: https://xdaforums.com/t/how-to-compile-a-c-program-for-android-using-the-android-ndk-cli-tools.4656126/

This section contains instructions to compile the libraries

libnl3, libpcap

and the binaries

nmon, cpio, bc, less, ngrep, openssl, vim, bvi, perl 5.40, NcFTP

Notes:

In the examples below gcc is used to create the binaries if possible. If gcc can not compile the source code, clang was used to create the binary.

All simple binaries mentioned in this examples are available here : https://bnsmb.de/files/public/Android/binaries_for_arm64/

The binaries are also partially available in Magisk Module - see the list of Magisk Modules below.

see also Compiling C programs in Android

see also How to compile Perl for Android

Before you re-invent the wheel, I recommend checking whether the tool you need is part of the repo https://github.com/Zackptg5/Cross-Compiled-Binaries-Android.
This repo contains binaries for Android and also scripts to build to the binaries from the source.

A very good source for compiler and linker options for compiling Unix programs und Android are the sources for Termux : https://github.com/termux/termux-packages/tree/master/packages

Examples for compiling C programs for Android using gcc or clang

How to compile nmon

Source: https://nmon.sourceforge.io/pmwiki.php

API="21" NDK="/data/develop/android/android-ndk-r14b" NDK_SYSROOT="${NDK}/platforms/android-${API}/arch-arm64" NDK_TOOLCHAIN= "${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64" NDK_BINDIR="${NDK_TOOLCHAIN}/bin" SYSROOT="/data/develop/android/android_root" ${NDK_BINDIR}/aarch64-linux-android-gcc-4.9 --sysroot=${NDK_SYSROOT} -I${SYSROOT}/usr/include -fPIE lmon16p.c -o nmon \ -L${SYSROOT}/usr/lib -pie -D arm64 -Wno-format-security -Wno-format -lncurses -O3 -Wall -D JFS -D GETUSER -D LARGEMEM -g

Update 12.08.2024

To create a static linked binary use:

${NDK_BINDIR}/aarch64-linux-android-gcc-4.9 --sysroot=${NDK_SYSROOT} -I${SYSROOT}/usr/include -fPIE lmon16p.c -o nmon \ -L${SYSROOT}/usr/lib -static -D arm64 -Wno-format-security -Wno-format -lncurses -O3 -Wall -D JFS -D GETUSER -D LARGEMEM -g

Note:

If the compiler complains about a missing fstab.h, create a dummy fstab.h:

touch ${SYSROOT}/usr/include/fstab.h

(Note: nmon does not list any infos about the filesystems using this work around)

nmon requires ncurses

How to compile cpio

Source: http://ftp.fau.de/gnu/cpio/cpio-2.9.tar.gz

export API="21" export NDK="/data/develop/android/android-ndk-r25.2" export NDK_TOOLCHAIN="${NDK}/toolchains/llvm/prebuilt/linux-x86_64" export NDK_BINDIR="${NDK_TOOLCHAIN}/bin" export NDK_SYSROOT="${NDK_TOOLCHAIN}/sysroot/" export SYSROOT="/data/develop/android/android_root" export CC="${NDK_BINDIR}/aarch64-linux-android${API}-clang" export AR="${NDK_BINDIR}/llvm-ar" export LD="${NDK_BINDIR}/ld" export CXX="${NDK_BINDIR}/aarch64-linux-android${API}-clang++" export RANLIB="${NDK_BINDIR}/llvm-ranlib" export STRIP="${NDK_BINDIR}/llvm-strip" export AS="${NDK_BINDIR}/aarch64-linux-android${API}-clang" export CFLAGS="--sysroot=${NDK_SYSROOT} -fPIE" export LDFLAGS="--sysroot=${NDK_SYSROOT} -pie" ./configure --prefix=/data/develop/android/android_root/ --host arm make

Notes:

bc does not need any additional library

How to compile bc

Source: http://ftp.fau.de/gnu/bc/bc-1.07.tar.gz

exportAPI="21" export NDK="/data/develop/android/android-ndk-r25.2" export NDK_TOOLCHAIN="${NDK}/toolchains/llvm/prebuilt/linux-x86_64" export NDK_BINDIR="${NDK_TOOLCHAIN}/bin" export NDK_SYSROOT="${NDK_TOOLCHAIN}/sysroot/" export SYSROOT="/data/develop/android/android_root" export CC="${NDK_BINDIR}/aarch64-linux-android${API}-clang" export AR="${NDK_BINDIR}/llvm-ar" export LD="${NDK_BINDIR}/ld" export CXX="${NDK_BINDIR}/aarch64-linux-android${API}-clang++" export RANLIB="${NDK_BINDIR}/llvm-ranlib" export STRIP="${NDK_BINDIR}/llvm-strip" export AS="${NDK_BINDIR}/aarch64-linux-android${API}-clang" export CFLAGS="--sysroot=${NDK_SYSROOT} -fPIE" export LDFLAGS="--sysroot=${NDK_SYSROOT} -pie" ./configure --host arm64 make
Notes: bc does not need any additional library

How to compile less

Source: http://ftp.fau.de/gnu/less/less-643.tar.gz

export API="21" export NDK="/data/develop/android/android-ndk-r14b" export NDK_SYSROOT="${NDK}/platforms/android-${API}/arch-arm64" exportNDK_TOOLCHAIN="${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64" export NDK_BINDIR="${NDK_TOOLCHAIN}/bin" export SYSROOT="/data/develop/android/android_root" export CC="${NDK_BINDIR}/aarch64-linux-android-gcc-4.9" export AR="${NDK_BINDIR}/aarch64-linux-android-ar" export LD="${NDK_BINDIR}/aarch64-linux-android-ld" export CXX="${NDK_BINDIR}/aarch64-linux-android-c++" export RANLIB="${NDK_BINDIR}/aarch64-linux-android-ranlib" export STRIP="${NDK_BINDIR}/aarch64-linux-android-strip" export AS="${NDK_BINDIR}/aarch64-linux-android-as" export CFLAGS="--sysroot=${NDK_SYSROOT} -fPIE -I${SYSROOT}/usr/include" export LDFLAGS="--sysroot=${NDK_SYSROOT} -pie -L${SYSROOT}/usr/lib" ./configure makeNote:

less requires ncurses

How to compile libnl3

Source: https://github.com/thom311/libnl/releases/download/libnl3_9_0/libnl-3.9.0.tar.gz

exportAPI="26" export NDK="/data/develop/android/android-ndk-r25.2" export NDK_TOOLCHAIN="${NDK}/toolchains/llvm/prebuilt/linux-x86_64" export NDK_BINDIR="${NDK_TOOLCHAIN}/bin" export NDK_SYSROOT="${NDK_TOOLCHAIN}/sysroot/" export SYSROOT="/data/develop/android/android_root" export CC="${NDK_BINDIR}/aarch64-linux-android${API}-clang" export AR="${NDK_BINDIR}/llvm-ar" export LD="${NDK_BINDIR}/ld" export CXX="${NDK_BINDIR}/aarch64-linux-android${API}-clang++" export RANLIB="${NDK_BINDIR}/llvm-ranlib" export STRIP="${NDK_BINDIR}/llvm-strip" export AS="${NDK_BINDIR}/aarch64-linux-android${API}-clang" export CFLAGS="--sysroot=${NDK_SYSROOT} -fPIC" export LDFLAGS="--sysroot=${NDK_SYSROOT} -pie" ./configure --prefix=/data/develop/android/android_root/usr --host="aarch64-linux-android" --disable-pthreads sed -i '/^struct in_addr/itypedef __be32 in_addr_t;' include/linux-private/linux/in.h makeCredits: https://github.com/thom311/libnl/issues/279

How to compile libpcap

Source: https://www.tcpdump.org/release/libpcap-1.10.1.tar.gz

exportAPI="26" export NDK="/data/develop/android/android-ndk-r25.2" export NDK_TOOLCHAIN="${NDK}/toolchains/llvm/prebuilt/linux-x86_64" export NDK_BINDIR="${NDK_TOOLCHAIN}/bin" export NDK_SYSROOT="${NDK_TOOLCHAIN}/sysroot/" export SYSROOT="/data/develop/android/android_root" export CC="${NDK_BINDIR}/aarch64-linux-android${API}-clang" export AR="${NDK_BINDIR}/llvm-ar" export LD="${NDK_BINDIR}/ld" export CXX="${NDK_BINDIR}/aarch64-linux-android${API}-clang++" export RANLIB="${NDK_BINDIR}/llvm-ranlib" export STRIP="${NDK_BINDIR}/llvm-strip" export AS="${NDK_BINDIR}/aarch64-linux-android${API}-clang" export CFLAGS="--sysroot=${NDK_SYSROOT} -fPIC -I/data/develop/android/android_root/usr/include" export LDFLAGS="--sysroot=${NDK_SYSROOT} -pie -L /data/develop/android/android_root/usr/lib"
./configure --prefix=/data/develop/android/android_root/usr --host="aarch64-linux-android" --with-pcap=/data/develop/android/android_root/ make
Note:

libpcap requires libnl3

How to compile ngrep

Source: https://github.com/jpr5/ngrep/archive/refs/heads/master.zip

exportAPI="26" export NDK="/data/develop/android/android-ndk-r25.2" export NDK_TOOLCHAIN="${NDK}/toolchains/llvm/prebuilt/linux-x86_64" export NDK_BINDIR="${NDK_TOOLCHAIN}/bin" export NDK_SYSROOT="${NDK_TOOLCHAIN}/sysroot/" export SYSROOT="/data/develop/android/android_root" export CC="${NDK_BINDIR}/aarch64-linux-android${API}-clang" export AR="${NDK_BINDIR}/llvm-ar" export LD="${NDK_BINDIR}/ld" export CXX="${NDK_BINDIR}/aarch64-linux-android${API}-clang++" export RANLIB="${NDK_BINDIR}/llvm-ranlib" export STRIP="${NDK_BINDIR}/llvm-strip" export AS="${NDK_BINDIR}/aarch64-linux-android${API}-clang" export CFLAGS="--sysroot=${NDK_SYSROOT} -fPIC -I/data/develop/android/android_root/usr/include -Wno-implicit-int -Wno-deprecated-non-prototype" export LDFLAGS="--sysroot=${NDK_SYSROOT} -pie -L /data/develop/android/android_root/usr/lib"
# create the Make files # ./configure --prefix=/data/develop/android/android_root/usr --host="aarch64-linux-android" # correct the make file for the regex library # sed -i -e "s#^CFLAGS =#CFLAGS=${CFLAGS} #g" regex-0.12/Makefile # ngrep uses some deprecated pcap functions (the ngrep source code was last updated 2017) # cp $SYSROOT//usr/include/pcap/pcap.h $SYSROOT//usr/include/pcap/pcap.h.org sed -i -e "s/PCAP_DEPRECATED.*//g" $SYSROOT//usr/include/pcap/pcap.h # there is an hardcoded include for /usr/include/pcap in the Makefile that must be removed # cp Makefile Makefile.org sed -i -e "s#-I/usr/include/pcap##g" Makefile make
Note:

ngrep requires libpcap

The ngrep binary is part of the Magisk Module with the DebugTools that is available on my homepage -- see the list of Magisk Modules below

How to compile OpenSSL

Source: https://github.com/openssl/openssl/releases/download/openssl-3.3.1/openssl-3.3.1.tar.gz

To compile OpenSSL for arm64 using the NDK only a few environment variables are necessary:

export TARGET=aarch64-linux-android export ANDROID_NDK_ROOT="/data/develop/android/android-ndk-r14b" export PATH=${ANDROID_NDK_ROOT}/toolchains/${TARGET}-4.9/prebuilt/linux-x86_64/bin:$PATH ./Configure android-arm64

Notes:

Make sure that no other environment variables for cross compiling are defined before calling the Configure script!

Check the file NOTES-ANDROID.md in the OpenSSL source for further info regarindg compiling OpenSSL for Android

To use "make install" to install the compiled OpenSSL libraries into the NDK use the additional parameter

--prefix=/data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64

android-24 is the API version used to compile the OpenSSL binaries; without the parameter to define the API version the Configure script from the OpenSSL source uses the latest API available in the used NDK, to print that version use the command

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/source/openssl-3.3.1 ] $ ls -ld /data/develop/android/android-ndk-r14b/platforms/android-*/arch-arm64 drwxr-xr-x. 3 xtrnaw7 xtrnaw7 4096 Mar 15 2017 /data/develop/android/android-ndk-r14b/platforms/android-21/arch-arm64 drwxr-xr-x. 3 xtrnaw7 xtrnaw7 4096 Mar 15 2017 /data/develop/android/android-ndk-r14b/platforms/android-22/arch-arm64 drwxr-xr-x. 3 xtrnaw7 xtrnaw7 4096 Mar 15 2017 /data/develop/android/android-ndk-r14b/platforms/android-23/arch-arm64 drwxr-xr-x. 3 xtrnaw7 xtrnaw7 4096 Mar 15 2017 /data/develop/android/android-ndk-r14b/platforms/android-24/arch-arm64 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/source/openssl-3.3.1 ] $

To use another API version use the additional parameter -D__ANDROID_API__=N for the Configure script.

A Magisk Module with this version of OpenSSL is available on my homepage -- see the list of Magisk Modules below

How to compile vim

Source: git clone https://github.com/vim/vim.git

# for static build: export CFLAGS="-static -O2" export LDFLAGS="-static -s" # for dynamic build: # # export CFLAGS='-O2 -fPIE -fPIC' # export LDFLAGS='-s -pie' # # for the ASUS Zenfone 8 use # export TARGET=aarch64-linux-android # Set this to your minSdkVersion. # export API=21 export NDK="/data/develop/android/android-ndk-r14b" export SYSROOT="${NDK}/platforms/android-${API}/arch-arm64" export CFLAGS="--sysroot=${SYSROOT} ${CFLAGS} -I/data/develop/android/sysroot/usr/include -I${SYSROOT}/usr/include " export LDFLAGS="--sysroot=${SYSROOT} ${LDFLAGS} -L/data/develop/android/sysroot/usr/lib -L${SYSROOT}/usr/lib " export CPPFLAGS="${CFLAGS}" export CC="${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-gcc-4.9" export AR=${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-gcc-ar export LD=${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-gcc-ld export CXX=${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-c++ export RANLIB=${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-ranlib export STRIP=${NDK}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/${TARGET}-strip export AS=${CC} ./configure --host ${TARGET} --prefix=/data/local/tmp \ --disable-nls \ --with-tlib=ncurses \ --without-x \ --enable-gui=no \ --enable-multibyte \ --enable-terminal \ remove_size \ ac_cv_sizeof_int=4 \ vim_cv_getcwd_broken=no \ vim_cv_memmove_handles_overlap=yes \ vim_cv_stat_ignores_slash=yes \ vim_cv_tgetent=zero \ vim_cv_terminfo=yes \ vim_cv_toupper_broken=no \ vim_cv_timer_create=yes \ vim_cv_tty_group=world make

The parameter for the configure script are taken from this script https://github.com/Zackptg5/Cross-Compiled-Binaries-Android/blob/master/build_script/build.sh

Notes:

vim requires libncurses, /data/develop/android/sysroot/usr/ is the directory tree with the required ncurses library in these instructions

The default directories for the vim config files are defined in the file /src/auto/pathdef.c:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/source/vim ] $ grep _dir ./src/auto/pathdef.c char_u *default_vim_dir = (char_u *)"/data/local/tmp/share/vim"; char_u *default_vimruntime_dir = (char_u *)""; [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/source/vim ] $

This file is created by the make file ; to change the values for the directory use configure parameter to define these variables:

default_vim_dir default_vimruntime_dir

---

A Magisk Module with this version of vim is available on my homepage -- see the list of Magisk Modules below

How to compile a static linked bvi

Source: http://sourceforge.net/projects/bvi/files/bvi/1.4.2/bvi-1.4.2.src.tar.gz

Prepare the config files for bvi (see How to compile a C program for Android using gcc for details)

cp config.h.in config.h.in.org sed -i -e "/#undef malloc/d" -e "/#undef realloc/d" malloc config.h.in

The result should look like this:
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/source/test/bvi-1.4.2 ] $ diff config.h.in config.h.in.org 134a135 > #undef malloc 139a141 > #undef realloc [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/source/test/bvi-1.4.2 ] $

# for static build: export CFLAGS="-static -O2" export LDFLAGS="-static -s -ffunction-sections -fdata-sections -Wl,--gc-sections " # for dynamic build: # # export CFLAGS='-O2 -fPIE -fPIC' # export LDFLAGS='-s -pie' # # for the ASUS Zenfone 8 use # export TARGET=aarch64-linux-android # Set this to your minSdkVersion. # export API=21 export NDK=/data/develop/android/android-ndk-r25.2 export TOOLCHAIN=${NDK}/toolchains/llvm/prebuilt/linux-x86_64 export CFLAGS="--sysroot ${TOOLCHAIN}/sysroot ${CFLAGS}" export LDFLAGS="--sysroot ${TOOLCHAIN}/sysroot ${LDFLAGS}" export CC=${TOOLCHAIN}/bin/$TARGET$API-clang export AR=${TOOLCHAIN}/bin/llvm-ar export AS=${CC} export CXX=${TOOLCHAIN}/bin/$TARGET$API-clang++ export LD=${TOOLCHAIN}/bin/ld export RANLIB=${TOOLCHAIN}/bin/llvm-ranlib export STRIP=${TOOLCHAIN}/bin/llvm-strip # include the additional sysroot directories with the files for the ncurses library # export CFLAGS="${CFLAGS} -w -I/data/develop/android/sysroot/usr/include -Wno-implicit-function-declaration -Wno-int-conversion" export LDFLAGS="${LDFLAGS} -L/data/develop/android/android_root/usr/lib" ./configure --host ${TARGET} --with-ncurses=/data/develop/android/sysroot/usr/ make

A Magisk Module with this version of bvi is available on my homepage -- see the list of Magisk Modules below

How to compile Perl 5.40

Source: https://www.cpan.org/src/5.0/perl-5.40.0.tar.gz

Perl 5.40 can be compiled using the latest Android SDK with gcc -- this is SDK version r14b.

The build configuration for creating the binaries for Android can be done using the Script Configure that is part of the tar archive with the source files for Perl.

Be aware that the Configure script needs a working connection to a phone - either via adb or ssh (see the file README.android from the tar archive).

In the instructions below a connection to the phone via adb is used.

Use these commands to compile Perl 5.40 wit the Android SDK:

# Define the API version to use # export API=24 # define the target CPU; for the ASUS Zenfone 8 or any other phone with arm64 CPU use: # export TARGETARCH="aarch64-linux-android" # define the target dir on the phone for the Perl installation # export PERL_TARGET_DIR_ON_THE_PHONE="/data/local/tmp/perl540" # define the NDK to use #export NDK="/data/develop/android/android-ndk-r14b" # add the NDK directory with the binaries to the PATH #export PATH="${NDK}/toolchains/${TARGETARCH}-4.9/prebuilt/`uname | tr '[A-Z]' '[a-z]'`-x86_64/bin:${PATH}" # create a temporary tool chain to compile Perl 5.40 in /tmp export ANDROID_TOOLCHAIN="/tmp/my-toolchain-${TARGETARCH}" export SYSROOT=${ANDROID_TOOLCHAIN}/sysroot # create the temporary tool chain directory #if [ ! -d "/tmp/my-toolchain-${TARGETARCH}/bin" ] ; then   $NDK/build/tools/make-standalone-toolchain.sh --platform=android-21   --install-dir="${ANDROID_TOOLCHAIN}" --toolchain="${TARGETARCH}-4.9" --arch=arm64 fi # define the s/n of the phone used -- if there is only one phone connected via this command can be used: #(this variable is used as parameter for the adb command in the Configure script) #DEVICE="$( adb devices | tail -2 | awk '{ print $1 }' )" # define the temporary directory on the phone used for compiling perl # export TARGETDIR="/data/local/tmp/perl_temp" # test adb access (and create the test directory on the phone ...) #adb -s $DEVICE shell "mkdir -p ${TARGETDIR} ; ls -ld ${TARGETDIR}" # Update 18.08.2024: Cleanup the temporary directory on the phone#
adb -s $DEVICE shell "rm -rf ${TARGETDIR}/* ; ls -la ${TARGETDIR}" # # create the target directory for the Perl binaries on the phone temporary on the PC # (the Configure script requires that these directories exist; the directories are only used when executing "make install") #mkdir -p "${PERL_TARGET_DIR_ON_THE_PHONE}/bin"mkdir -p "${PERL_TARGET_DIR_ON_THE_PHONE}/../bin"# start the Configure script from Perl - note: Do NOT use the parameter "-d -e" ; using these parameter the Configure script always fails# And I recommend not to use the parameter "-s" to see what's going on (-s = silent run) # ./Configure -Dprefix="${PERL_TARGET_DIR_ON_THE_PHONE}" -Dusedevel -Dusecrosscompile -Dtargetrun=adb -Dcc=${TARGETARCH}-gcc -Dsysroot="${SYSROOT}" -Dtargetdir="${TARGETDIR}" -Dtargethost=${DEVICE} \   -Dccflags=" -fPIE -pie --std=gnu99 --sysroot=${SYSROOT} " \   -Dldflags="-pie --sysroot=${SYSROOT} -L${ANDROID_TOOLCHAIN}/sysroot/usr/lib " \
   -Accflags=" -DPERL_LC_ALL_USES_NAME_VALUE_PAIRS -DPERL_LC_ALL_SEPARATOR -DPERL_LC_ALL_CATEGORY_POSITIONS_INIT"
The Configure script asks a lot of questions - for most of them the default value can be confirmed.

The only exception is this question:

Checking for GNU cc in disguise and/or its version number..."./try": error: Android 5.0 and later only support position-independent executables (-fPIE). You are not using GNU cc../Configure: line 4727: ./try: cannot execute binary file: Exec format error./Configure: line 4727: ./try: cannot execute binary file: Exec format error./Configure: line 4727: ./try: cannot execute binary file: Exec format errorYour C compiler doesn't seem to be able to compile C99 code Do you really want to continue? [n]

Just ignore the error and answer "y".

Reason:

The Configure script does not use the correct parameter for the gcc to create the test binary. Therefor the gcc successfully creates the test binary without error but the binary can not be executed in the Android OS.

So far I have not found out how to define the gcc parameters for this test compilation: The parameters for Configure script are not used here; the file hints/linux-android.sh is not yet read, and environment variables are also ignored as far as I can see.

To avoid this error you can edit the Configure script: see https://dev.to/jccr/compiling-perl-for-android-12a4 for details.

Another question for which you might want to not use the default value is this one:

Do you want to use a version number suffix for installed binaries? [y] n

If using the default (y) the Configure script creates a perl executable called "perl5.40.0" instead of "perl".

Use

make

to create the binaries.

To create the directory structure for Perl later on the phone execute on the PC:
make install

Now create a zip file or archive from the directory /data/local/tmp/perl540 and copy it to the phone

To cleanup the temporary directories used use these commands

# delete the temporary tool chain # rm -rf "${ANDROID_TOOLCHAIN}" # delete the directory on the phone with the temporary files # adb shell rm -rf "${TARGETDIR}"

Notes

The config.sh file used to build Perl 5.40 using the Android SDK r14b is config.sh_for_perl_5.40_in_Android_sdk_r14b.

These instructions can also be used to compile Perl 5.38

see the section How to compile Perl for Android for instructions to compile Perl 5.38 in a Termux session

Before starting a make test you must delete two lines for non-existent files from the file Makefile.SH (the files for the Pod-Parser are not part of the standard Perl anymore):
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/source/perl-5.40.0 ] $ cp Makefile.SH Makefile.SH.org [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/source/perl-5.40.0 ] $ sed -i -e "/.*Pod-Parser.*/d" Makefile.SH[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/source/perl-5.38.0 ] $ diff Makefile.SH Makefile.SH.org1559a1560>     $to cpan/Pod-Parser/*1565a1567>     $to cpan/Pod-Parser/lib/*[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/source/perl-5.38.0 ] $

The log file of "make test" for the Perl 5.40 executables created is make_test_for_perl540.txt.

---

A Magisk Module with this version of Perl is available on my homepage -- see the list of Magisk Modules below

How to compile NcFTP

Source: https://www.ncftp.com/public_ftp/ncftp/ncftp-3.2.7-src.tar.gz

Download and unpack the tar file

export CFLAGS="-static -O2 " # -fPIC export LDFLAGS="-static -s " # -ffunction-sections -fdata-sections -Wl,--gc-sections "## for the ASUS Zenfone 8 or other phones with ARM64 CPU use:#export TARGET=aarch64-linux-androidexport TOOLS_IDENTIFIER=aarch64-linux-androidexport TARGET_ROOT=/data/develop/android/sysroot# Set this to your minSdkVersion.# (newer APIs do not work)#export API=${API:=26}export NDK=/data/develop/android/android-ndk-r25.2export ANDROID_NDK_HOME=${NDK}export TOOLCHAIN=${NDK}/toolchains/llvm/prebuilt/linux-x86_64export SYSROOT="${TOOLCHAIN}/sysroot"export CFLAGS="--sysroot ${SYSROOT} ${CFLAGS}"export LDFLAGS="--sysroot ${SYSROOT} ${LDFLAGS}"export CC=${TOOLCHAIN}/bin/$TOOLS_IDENTIFIER$API-clangexport AR=${TOOLCHAIN}/bin/llvm-arexport AS=${CC}export CXX=${TOOLCHAIN}/bin/$TOOLS_IDENTIFIER$API-clang++export LD=${TOOLCHAIN}/bin/ldexport RANLIB=${TOOLCHAIN}/bin/llvm-ranlibexport STRIP=${TOOLCHAIN}/bin/llvm-strip# include the additional sysroot directories with the files for the ncurses library#export CFLAGS="${CFLAGS} -w -I${TARGET_ROOT}/usr/include -I${SYSROOT}/usr/include -Wno-implicit-function-declaration -Wno-int-conversion"export LDFLAGS="${LDFLAGS} -L${TARGET_ROOT}/usr/lib -L${SYSROOT}/lib "./configure --host ${TARGET} --target=${TARGET} --prefix=/system --without-curses --without-ncurses

----

Tool chain HowTos

Print all gcc builtin Macros

$CC -dM -E - < /dev/null

e.g.

xtrnaw7@fedora01:/data/develop/android/source $ $CC -dM -E - < /dev/null | grep ANDROID #define __ANDROID_API__ __ANDROID_MIN_SDK_VERSION__ #define __ANDROID_MIN_SDK_VERSION__ 26 #define __ANDROID__ 1 xtrnaw7@fedora01:/data/develop/android/source $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/android-ndk-r26b/toolchains/llvm/prebuilt/linux-x86_64/bin ] $ ./x86_64-linux-android34-clang -dM -E - < /dev/null | grep ANDROID #define __ANDROID_API__ __ANDROID_MIN_SDK_VERSION__ #define __ANDROID_MIN_SDK_VERSION__ 34 #define __ANDROID__ 1 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/android-ndk-r26b/toolchains/llvm/prebuilt/linux-x86_64/bin ] $

Print gcc build options

To print the GCC build options use
gcc -v
or
gcc -###

Examples

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/android-ndk-r14b/toolchains/x86_64-4.9/prebuilt/linux-x86_64/bin ] $ ./x86_64-linux-android-gcc -v Using built-in specs. COLLECT_GCC=./x86_64-linux-android-gcc COLLECT_LTO_WRAPPER=/data/develop/android/android-ndk-r14b/toolchains/x86_64-4.9/prebuilt/linux-x86_64/bin/../libexec/gcc/x86_64-linux-android/4.9.x/lto-wrapper Target: x86_64-linux-android Configured with: /usr/local/google/buildbot/src/android/gcc/toolchain/build/../gcc/gcc-4.9/configure --prefix=/tmp/ca8b1ea984a04012a16bb19760bc0d9a --target=x86_64-linux-android --host=x86_64-linux-gnu --build=x86_64-linux-gnu --with-gnu-as --with-gnu-ld --enable-languages=c,c++ --with-gmp=/buildbot/tmp/build/toolchain/temp-install --with-mpfr=/buildbot/tmp/build/toolchain/temp-install --with-mpc=/buildbot/tmp/build/toolchain/temp-install --with-cloog=/buildbot/tmp/build/toolchain/temp-install --with-isl=/buildbot/tmp/build/toolchain/temp-install --with-ppl=/buildbot/tmp/build/toolchain/temp-install --disable-ppl-version-check --disable-cloog-version-check --disable-isl-version-check --enable-cloog-backend=isl --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --disable-libssp --enable-threads --disable-nls --disable-libmudflap --disable-libgomp --disable-libstdc__-v3 --disable-sjlj-exceptions --disable-shared --disable-tls --disable-libitm --with-arch=x86-64 --with-tune=intel --with-fpmath=sse --with-multilib-list=m32,m64,mx32 --enable-bionic-libs --enable-libatomic-ifuncs=no --enable-initfini-array --disable-nls --prefix=/tmp/ca8b1ea984a04012a16bb19760bc0d9a --with-sysroot=/tmp/ca8b1ea984a04012a16bb19760bc0d9a/sysroot --with-binutils-version=2.25 --with-mpfr-version=3.1.1 --with-mpc-version=1.0.1 --with-gmp-version=5.0.5 --with-gcc-version=4.9 --with-gdb-version=none --with-gxx-include-dir=/tmp/ca8b1ea984a04012a16bb19760bc0d9a/include/c++/4.9.x --with-bugurl=http://source.android.com/source/report-bugs.html --enable-languages=c,c++ --disable-bootstrap --enable-plugins --enable-libgomp --enable-gnu-indirect-function --disable-libcilkrts --disable-libsanitizer --enable-gold --enable-threads --enable-eh-frame-hdr-for-static --enable-graphite=yes --with-isl-version=0.11.1 --with-cloog-version=0.18.0 --program-transform-name='s&^&x86_64-linux-android-&' --enable-gold=default Thread model: posix gcc version 4.9.x 20150123 (prerelease) (GCC) [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/android-ndk-r14b/toolchains/x86_64-4.9/prebuilt/linux-x86_64/bin ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/android-ndk-r14b/toolchains/x86_64-4.9/prebuilt/linux-x86_64/bin ] $ ./x86_64-linux-android-gcc -### -E 2>&1 | grep "Configured with" | sed 's/--/\n--/g' Configured with: /usr/local/google/buildbot/src/android/gcc/toolchain/build/../gcc/gcc-4.9/configure --prefix=/tmp/ca8b1ea984a04012a16bb19760bc0d9a --target=x86_64-linux-android --host=x86_64-linux-gnu --build=x86_64-linux-gnu --with-gnu-as --with-gnu-ld --enable-languages=c,c++ --with-gmp=/buildbot/tmp/build/toolchain/temp-install --with-mpfr=/buildbot/tmp/build/toolchain/temp-install --with-mpc=/buildbot/tmp/build/toolchain/temp-install --with-cloog=/buildbot/tmp/build/toolchain/temp-install --with-isl=/buildbot/tmp/build/toolchain/temp-install --with-ppl=/buildbot/tmp/build/toolchain/temp-install --disable-ppl-version-check --disable-cloog-version-check --disable-isl-version-check --enable-cloog-backend=isl --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --disable-libssp --enable-threads --disable-nls --disable-libmudflap --disable-libgomp --disable-libstdc__-v3 --disable-sjlj-exceptions --disable-shared --disable-tls --disable-libitm --with-arch=x86-64 --with-tune=intel --with-fpmath=sse --with-multilib-list=m32,m64,mx32 --enable-bionic-libs --enable-libatomic-ifuncs=no --enable-initfini-array --disable-nls --prefix=/tmp/ca8b1ea984a04012a16bb19760bc0d9a --with-sysroot=/tmp/ca8b1ea984a04012a16bb19760bc0d9a/sysroot --with-binutils-version=2.25 --with-mpfr-version=3.1.1 --with-mpc-version=1.0.1 --with-gmp-version=5.0.5 --with-gcc-version=4.9 --with-gdb-version=none --with-gxx-include-dir=/tmp/ca8b1ea984a04012a16bb19760bc0d9a/include/c++/4.9.x --with-bugurl=http://source.android.com/source/report-bugs.html --enable-languages=c,c++ --disable-bootstrap --enable-plugins --enable-libgomp --enable-gnu-indirect-function --disable-libcilkrts --disable-libsanitizer --enable-gold --enable-threads --enable-eh-frame-hdr-for-static --enable-graphite=yes --with-isl-version=0.11.1 --with-cloog-version=0.18.0 --program-transform-name='s&^&x86_64-linux-android-&' --enable-gold=default [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/android-ndk-r14b/toolchains/x86_64-4.9/prebuilt/linux-x86_64/bin ] $

Print the gcc version

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/android-ndk-r14b/toolchains/x86_64-4.9/prebuilt/linux-x86_64/bin ] $ ./x86_64-linux-android-gcc --version x86_64-linux-android-gcc (GCC) 4.9.x 20150123 (prerelease) Copyright (C) 2014 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/android-ndk-r14b/toolchains/x86_64-4.9/prebuilt/linux-x86_64/bin ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/android-ndk-r14b/toolchains/x86_64-4.9/prebuilt/linux-x86_64/bin ] $ ./x86_64-linux-android-gcc -dumpversion 4.9.x [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/android-ndk-r14b/toolchains/x86_64-4.9/prebuilt/linux-x86_64/bin ] $

Trouble Shooting

A list of common problems and solutions for compiling with the Android NDK is here: https://developer.android.com/ndk/guides/common-problems

If one of the standard include files is missing, check the sysroot of a newer API version to see if it contains this file.

To avoid problems because of invalid settings for the script ./configure it's recommended to delete all cache files before starting ./configure, e.g:

xtrnaw7@fedora01:/data/develop/android/source/ngrep-1.47$ find . -name config.cache -ls 19970 1 -rw-r--r-- 1 xtrnaw7 xtrnaw7 0 Feb 19 08:19 ./regex-0.12/config.cache xtrnaw7@fedora01:/data/develop/android/source/ngrep-1.47$ # delete the config.cache files # find . -name config.cache -exec rm {} \; # check: find . -name config.cache -ls

ERROR: fatal error:sys/fcntl.h:No such file or directory

Solution:

Use the headers

#include <fcntl.h> #include <asm-generic/fcntl.h

ERROR: undefined reference to `getsubopt'

Solution:

Use a newer API version: getsubopt is defined in libc.so for android-26 and newer, but not for earlier platforms:

trnaw7@fedora01:/data/develop/android/source/libnl-3.9.0$ cd /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/ xtrnaw7@fedora01:/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib$ xtrnaw7@fedora01:/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib$ for i in ./aarch64-linux-android/*/libc.so ; do echo "** $i:" ; nm $i | grep getsubopt ; done ** ./aarch64-linux-android/21/libc.so: ** ./aarch64-linux-android/22/libc.so: ** ./aarch64-linux-android/23/libc.so: ** ./aarch64-linux-android/24/libc.so: ** ./aarch64-linux-android/26/libc.so: 000000000001c440 T getsubopt ** ./aarch64-linux-android/27/libc.so: 000000000001c440 T getsubopt ** ./aarch64-linux-android/28/libc.so: 000000000001e450 T getsubopt ** ./aarch64-linux-android/29/libc.so: 000000000001e450 T getsubopt ** ./aarch64-linux-android/30/libc.so: 000000000001f450 T getsubopt ** ./aarch64-linux-android/31/libc.so: 0000000000020450 T getsubopt ** ./aarch64-linux-android/32/libc.so: 0000000000020450 T getsubopt ** ./aarch64-linux-android/33/libc.so: 0000000000020450 T getsubopt xtrnaw7@fedora01:/data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib$

ERROR: error: undefined reference to 'stderr'

Solution:

The most likely reason for error messsages like this

scanner.c:(.text+0x8c): undefined reference to `stdin' scanner.c:(.text+0x90): undefined reference to `stdin' scanner.c:(.text+0xbc): undefined reference to `stdout' scanner.c:(.text+0xc0): undefined reference to `stdout' /data/develop/android/android_root/usr/lib/libpcap.a(scanner.o): In function `yy_fatal_error': scanner.c:(.text+0x259c): undefined reference to `stderr' scanner.c:(.text+0x25a0): undefined reference to `stderr'
is that you built for android-23+ but the lib used has a minSdkVersion below that.

Credits: https://stackoverflow.com/questions/51767214/android-studio-with-ndk-link-error-undefined-reference-to-stderr

ERROR: faddrs.h: No such file or directory

Solution:

Use a more current API -- the file ifaddrs.h was added to Android in API24. The file is only available in the sysroot for CLANG.

ERROR: ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]

Solution:

Add the option -Wno-implicit-function-declaration to the environment variable CFLAGS before executing ./configure:

export CFLAGS="$CFLAGS -Wno-implicit-function-declaration"

ERROR: "cannot find crtbegin_dynamic.o: No such file or directory" or "cannot find crtend_android.o: No such file or directory"

Solution:

Add the parameter for the sysroot (specifying the library path with the -L parameter is NOT sufficient here!) e.g:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/source/perl-5.40.0 ] $ aarch64-linux-android-gcc --sysroot=/tmp/my-toolchain-aarch64-linux-android/sysroot/ -o perl -pie -L/tmp/my-toolchain-aarch64-linux-android/sysroot/usr/lib -pie perlmain.o libperl.a `cat ext.libs` -ldl -lm -lc -llog -L/tmp/my-toolchain-aarch64-linux-android/sysroot/usr/lib[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/source/perl-5.40.0 ] $

ERROR: error: "./nmap": executable's TLS segment is underaligned: alignment is 8, needs to be at least 64 for ARM64 Bionic

Executing a binary on the phone aborts with an error message like this:

ASUS_I006D:/data/local/tmp $ ./nmaperror: "./nmap": executable's TLS segment is underaligned: alignment is 8, needs to be at least 64 for ARM64 BionicAborted 134|ASUS_I006D:/data/local/tmp $
Solution:

Either add the CFLAGS

-ffunction-sections -fdata-sections -Wl,--gc-sections

and rebuild the binary or correct the alignment using the script align_fix.py, e.g.:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/source/nmap-7.93 ] $ ../align_fix.py ./nmapFound TLS segment with align = 8TLS segment is underaligned, patching[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/source/nmap-7.93 ]

The source for the script align_fix.py was copied from this webpage: https://stackoverflow.com/questions/72496162/how-to-cross-compile-kernel-exec-tools-for-android

---

History of this entry

17.02.2024 /bs

initial release

18.02.2024 /bs

added the Trouble Shooting section
added the instructions to compile libnl3
added the instructions to compile libpcap

19.02.2024 /bs

corrected the instructions to compile libnl3: API must be 26 or greater
corrected the instructions to compile libpcap: API must be 26 or greater; the -I option was missing in the CFLAGS; the -L option was missing the LDFLAGS
added the instructions to compile ngrep

09.03.2024 /bs

updated the Trouble Shooting section

01.04.2024 /bs

updated the Tool chains HowTos

05.08.2024

added the instructions to comile OpenSSL

12.08.2024

added the instructions to compile static and dynamic vim executables.
added the instructions to compile a static bvi executable

16.08.2024

add the instructions to compile Perl 5.40

18.08.2024

added some more info to the instructions to compile Perl 5.40 using the Android SDK
rerun "make test" for Perl 5.40 and attached the new log file; the log file for "perl harness" was removed (the test failed because the temporary directory on the phone was not empty)

19.08.2024

added the solution to fix the alignment error

25.08.2024

added the instructions to compile NcFTP

Compiling C programs in Android

URL: https://xdaforums.com/t/how-to-compile-c-programs-in-android.4683687/

Compiling C programs in Android

To compile C or C++ programs under Android, I recommend installing the Linux environment Termux on the phone. Termux is basically a normal Linux environment that runs on the phone and compiling programs in Termux is more or less just like compiling in Linux on the PC (see below for some links with further infos about Termux).

To compile an existing Linux program the standard Linux tools like ./configure and make can be used in a Termux session. If a program complains about a missing library or tool just install it using

pkg install <packagename>

Example ./configure command to create the config files for building a Linux binary in a Termux session:

./configure --prefix=/data/data/com.termux/files CFLAGS="-Wno-implicit-function-declaration -Wno-int-conversion"

To create a binary that should run in an adb session and not in a Termux session add the parameters

--prefix=/data/local/tmp --datarootdir=/data/local/tmp

to the configure command (note that the Termux user can not access the directory /data/local/tmp without changing the permissions).

In principle, most Termux executables can also be used in an adb session. However, the user shell used in an adb session may not access the files in the directories used for Termux. In an adb session, root access is required to access the Termux files. Note that files created by the root user (implicitly or explicitly) in the directories used by Termux can no longer be used by the Termux user. Therefore, access to Termux files in an adb session should be avoided. Instead, copy the required files from the Termux directories to a directory available to the user shell (e.g. /data/local/tmp; but be aware that the directory /data/data/com.termux is hardcoded in some of the executables and therefor the executables will not work without access to that directory tree).

(see How to compile Perl for Android for an example)

Compiling programs for Android using Termux is easy doing but IMHO it's not fun to do this using the virtual keyboard of the phone (at least on my Zenfone 8 ...)
(A tool like scrcpy can be used to mirror the screen of the phone to the PC but this is still only a workaround)

Fortunately Termux contains an sshd and supports access via ssh.

For details how to configure access to Termux via ssh see here:

https://glow.li/posts/run-an-ssh-server-on-your-android-with-termux/

for details about how to configure access to Termux via adb see here (see access via adb is also using the sshd from Termux)

https://glow.li/posts/access-termux-via-usb/

To automatically start the sshd from Termux after opening the Termux App on the phone create a .profile like this:

cat /data/data/com.termux/files/home/.profile

exec >/data/data/com.termux/files/home/ssh_start.log 2>&1 CUR_USER=$( id -u -n ) ps -ef | grep "^${CUR_USER}" | grep -v grep | grep sshd >/dev/null if [ $? -ne 0 ] ; then echo "Starting the sshd ..." sshd ps -ef | grep sshd else echo "sshd already running" fi

To start the sshd from Termux automatically after the reboot of the phone create a Magisk start script to start the Termux App:

cat /data/adb/service.d/start_sshd_from_termux

if ! tty -s ; then exec >"/data/cache/${0##*/}.log" 2>&1 fi TIMEOUT=60 i=0 echo "Waiting up to ${TIMEOUT} seconds for /data/data/com.termux ..." while [ $i -lt ${TIMEOUT} ] ; do ls -ld /data/data/com.termux 2>/dev/null && break sleep 1 let i=i+1 printf "." done printf "\n" echo "Wait time: $i seconds "
set -x
ls -lZ /data/data/com.termux # # start the Termux App to init the environment am start com.termux/.app.TermuxActivity # the sshd must be started from within the Termux app -> add the start command for the sshd to the file ~/.profile for the Termux User # # check that the sshd is running sleep 3 ps -ef | grep sshd
Note:

Starting the Termux sshd from outside of the Termux app works also but only with an unusable, crippled environment in the ssh sessions.

Using both scripts the sshd from Termux starts automatically after every boot of the phone ; to access the phone via ssh use this command

ssh -p 8022 <IP_of_the_phone>

Example:

[ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ ssh -p 8022 192.168.1.148 "uname -a ; id" Linux localhost 5.4.147-Omni-qgki-perf-gf532250e9298 #1 SMP PREEMPT Thu Jul 25 17:09:09 UTC 2024 aarch64 Android uid=10149(u0_a149) gid=10149(u0_a149) groups=10149(u0_a149),3003(inet),9997(everybody),20149(u0_a149_cache),50149(all_a149) [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $

Notes

I just found out after implementing the method to automatically start the sshd from Termux described above , that the start of the sshd from Termux can also be configured via Termux services and Termux Boot - see :

https://www.reddit.com/r/termux/comments/12fp1xc/start_sshd_process_from_bashrc_or_zshrc_on_termux/
https://wiki.termux.com/wiki/Termux-services
https://wiki.termux.com/wiki/Termux:Boot

Termux is available at F-Droid: https://f-droid.org/packages/com.termux/.
Termux is also available in the Playstore but I recommend the version from F-Droid (but you should consider supporting the developer -- see https://termux.dev/en/donate)

The homepage for Termux is https://termux.dev/en/

There is also a Wiki with documentation for Termux: https://wiki.termux.com/

See here for instructions to build new binaries for Termux: https://github.com/termux/termux-packages

repo for the Termux app: https://github.com/termux/termux-app

Termux uses these directories on the phone:

Directory	Contents	Comment
/data/data/com.termux/files	base directory for the files used by Termux
/data/data/com.termux/files/usr	binaries, config files, etc: `ASUS_I006D:/ # ls /data/data/com.termux/files/usr/ bin etc include lib libexec share tmp var ASUS_I006D:/`
/data/data/com.termux/files/home/	writable home directory for Termux

History of this entry

25.07.2024

initial release

27.07.2024

added startup script examples to automatically start the Termux sshd

Compiling C programs in Android using gcc

URL: https://xdaforums.com/t/guide-compiling-c-programs-in-android-using-gcc.4695529/

Compiling C programs in Android using gcc

There is a Magisk Module with a GCC toolchain for Android that can be used to compile C programs in Android sessions: https://github.com/Googlers-Repo/gcc/.

I've tested it and IMHO the GCC toolchain is very useful for compiling some of the good old Unix programs for the Android OS without having to deal with cross-compiling issues on the PC.

Find below some hints for using that GCC toolchain to compile C programs in Android.

The Magisk Module can be installed using the instructions in the repository https://github.com/Googlers-Repo/gcc/, via the Magisk Module Repo Loader, the Magisk GUI, or the magisk binary.

The Magisk Module requires the Magisk Module for mksh and adds the directories with the tools from the GCC toolchain to the variable PATH via mksh profile:

shell@localhost:/data/mkuser/home/shell └─$ cat /system/etc/mkshrc.d/gcc_on_android.sh export PATH="$PATH:/system/usr/share/gcc/bin:/system/usr/share/gcc/aarch64-linux-android/bin:/system/usr/share/gcc/libexec/gcc/aarch64-linux-android/10.2.0"┌shell@localhost:/data/mkuser/home/shell └─$ ┌shell@localhost:/data/mkuser/home/shell └─$ echo $PATH /data/mkuser/usr/bin:/data/mkuser/home/shell/.local/bin:/system/bin:/sbin:/vendor/bin:/system/sbin:/system/xbin:/system/product/bin:/system/usr/share/bin-get/bin:/system/system_ext/bin:/system/usr/share/gcc/bin:/system/usr/share/gcc/aarch64-linux-android/bin:/system/usr/share/gcc/libexec/gcc/aarch64-linux-android/10.2.0 ┌shell@localhost:/data/mkuser/home/shell └─$ ┌shell@localhost:/data/mkuser/home/shell └─$ gcc --version aarch64-linux-android-gcc (GCC) 10.2.0 Copyright (C) 2020 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. ┌shell@localhost:/data/mkuser/home/shell └─$ ┌shell@localhost:/data/local/tmp/develop └─$ gcc -v Using built-in specs. COLLECT_GCC=aarch64-linux-android-gcc Target: aarch64-linux-android Configured with: ../configure --host=aarch64-linux-gnu --target=aarch64-linux-android --enable-static --disable-shared --enable-languages=c,c++ --enable-initfini-array --disable-lto --disable-libquadmath --disable-multilib --disable-libgomp --disable-libmudflap --enable-target-optspace --enable-threads --disable-libatomic --disable-tls --disable-nls --disable-sjlj-exceptions --disable-libstdc++-v3 --disable-libsanitizer --disable-plugins --disable-libgcc --disable-libssp --disable-docs --disable-libitm --with-gnu-as --with-gnu-ld --prefix=/home/n0n3m4/Desktop/C4droid/prog/c4droid/OutDir --with-mpc=/home/n0n3m4/Desktop/C4droid/prog/c4droid/mpc --with-mpfr=/home/n0n3m4/Desktop/C4droid/prog/c4droid/mpfr --with-gmp=/home/n0n3m4/Desktop/C4droid/prog/c4droid/gmp Thread model: posix Supported LTO compression algorithms: zlib gcc version 10.2.0 (GCC) ┌shell@localhost:/data/local/tmp/develop └─$

The libraries in the GCC toolchain are from the Android API version 21:

┌shell@localhost:/data/local/tmp/develop└─$ file bvi-1.4.2/bvi bvi-1.4.2/bvi: ELF shared object, 64-bit LSB arm64, dynamic (/system/bin/linker64), for Android 21, built by NDK r17c (4988734), not stripped┌shell@localhost:/data/local/tmp/develop└─$

The directory tree with the standard include files and libraries in the GCC toolchain is

/system/usr/share/gcc/aarch64-linux-android

To compile a C program using the GCC toolchain these flags for the compiler and linker are necessary:
export LDFLAGS="-pie" export CFLAGS="-fPIE"

I also recommend to create a separate sysroot for the compiled binaries and libraries :

mkdir -p /data/local/tmp/develop/sysroot/ && \ cd/data/local/tmp/develop/sysroot/ && \
mkdir -p usr usr/bin usr/sbin usr/include usr/lib usr/lib64 usr/share&& \ { ln -s usr/lib ./lib ln -s usr/lib64 ./lib64 ln -s usr/bin ./bin ln -s usr/sbin ./sbin }
Now you can use the configure script like this to create the make files for compiling a Unix tool:

export SYSROOT=/system/usr/share/gcc/aarch64-linux-androidexport TARGET_ROOT=/data/local/tmp/develop/sysrootexport LDFLAGS="-pie -L${TARGET_ROOT}/usr/lib" export CFLAGS="-fPIE -I${TARGET_ROOT}/usr/include"export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${TARGET_ROOT}/usr/lib
export PATH=${TARGET_ROOT}/usr/bin:${PATH} ./configure --prefix ${TARGET_ROOT}

use

make

to start the compile process

If necessary, add the parameter for the sysroot. Example to compile nmon:

# create a dummy fstab.h file for nmon # touch $TARGET_ROOT/usr/include/fstab.h gcc --sysroot=${SYSROOT} ${CFLAGS} ${LDFLAGS} lmon16q.c -o nmon -Wno-format-security -Wno-format -lncurses -O3 -Wall -D JFS -D GETUSER -D LARGEMEM -g

Example output

┌shell@localhost:/data/local/tmp/develop └─$ gcc --sysroot=${SYSROOT} ${CFLAGS} ${LDFLAGS} lmon16q.c -o nmon -Wno-format-security -Wno-format -lncurses -O3 -Wall -D JFS -D GETUSER -D LARGEMEM -g ┌shell@localhost:/data/local/tmp/develop └─$ ls -l nmon -rwxrwxrwx 1 shell shell 751512 2024-09-26 17:55 nmon ┌shell@localhost:/data/local/tmp/develop └─$ ┌shell@localhost:/data/local/tmp/develop └─$ file nmon nmon: ELF shared object, 64-bit LSB arm64, dynamic (/system/bin/linker64), for Android 21, built by NDK r17c (4988734), not stripped ┌shell@localhost:/data/local/tmp/develop └─$ ┌shell@localhost:/data/local/tmp/develop └─$ ldd $PWD/nmon linux-vdso.so.1 => [vdso] (0x7612a8e000) libm.so => /apex/com.android.runtime/lib64/bionic/libm.so (0x760db03000) libc.so => /apex/com.android.runtime/lib64/bionic/libc.so (0x760c854000) libdl.so => /apex/com.android.runtime/lib64/bionic/libdl.so (0x760db79000) ┌shell@localhost:/data/local/tmp/develop └─$ ┌shell@localhost:/data/local/tmp/develop └─$ ./nmon --help nmon: invalid option -- - Hint for nmon version 16q Full Help Info : nmon -h On-screen Stats: nmon Data Collection: nmon -f [-s <seconds>] [-c <count>] [-t|-T] Capacity Plan : nmon -x Interactive-Mode: Read the Welcome screen & at any time type: "h" for more help Type "q" to exit nmon For Data-Collect-Mode -f Must be the first option on the line (switches off interactive mode) Saves data to a CSV Spreadsheet format .nmon file in then local directory Note: -f sets a defaults -s300 -c288 which you can then modify Further Data Collection Options: -s <seconds> time between data snapshots -c <count> of snapshots before exiting -t Includes Top Processes stats (-T also collects command arguments) -x Capacity Planning=15 min snapshots for 1 day. (nmon -ft -s 900 -c 96) ---- End of Hints ┌shell@localhost:/data/local/tmp/develop └─$

For source code trees that are prepared for the use of configure but do not have a configure script (-> the file configure.ac exists but not the file configure), use the autoconf tool to create the configure script; the command to create the configure script is:

autoconf -fi

If autoconf ends with an error like this:

shell@localhost:/data/local/tmp/develop/unrar-free└─$ autoconf -ficonfigure.ac:6: error: possibly undefined macro: AM_INIT_AUTOMAKE If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation.┌shell@localhost:/data/local/tmp/develop/unrar-free└─$
execute the tool aclocal once and restart autoconf

aclocalautoconf -fi

A Magisk Module with autoconf (including the necessary tools for autoconf) is available here: autoconf_2.72.zip

Trouble Shooting

On some ROMs the install script for the Magisk Module with the GCC toolchain does not configure the execute permission for the wrapper scripts used in the Magisk Module. Therefor check the permissions for the scripts after installing the Magisk Module and correct them if necessary:

┌root@localhost:/└─#ls -l /data/adb/modules_update/gcc_on_android/system/usr/share/gcc/aarch64-linux-android/bintotal 14432-rwxr-xr-x 1 root shell 835784 2024-09-26 15:32 ar-rwxr-xr-x 1 root shell 1393248 2024-09-26 15:32 as-rw-r--r-- 1 root root 48 2024-09-26 15:32 cc-rwxr-xr-x 1 root shell 8082896 2024-09-26 15:32 cmake-rw-r--r-- 1 root root 48 2024-09-26 15:32 g++-rw-r--r-- 1 root root 48 2024-09-26 15:32 gcc-rwxr-xr-x 1 root shell 1807376 2024-09-26 15:32 ld-rwxr-xr-x 1 root shell 824400 2024-09-26 15:32 nm-rwxr-xr-x 1 root shell 835784 2024-09-26 15:32 ranlib-rwxr-xr-x 1 root shell 956800 2024-09-26 15:32 strip┌root@localhost:/└─#chmod 755 /data/adb/modules_update/gcc_on_android/system/usr/share/gcc/aarch64-linux-android/bin/*┌root@localhost:/└─# ls -l /data/adb/modules_update/gcc_on_android/system/usr/share/gcc/aarch64-linux-android/bintotal 14432-rwxr-xr-x 1 root shell 835784 2024-09-26 15:32 ar-rwxr-xr-x 1 root shell 1393248 2024-09-26 15:32 as-rwxr-xr-x 1 root root 48 2024-09-26 15:32 cc-rwxr-xr-x 1 root shell 8082896 2024-09-26 15:32 cmake-rwxr-xr-x 1 root root 48 2024-09-26 15:32 g++-rwxr-xr-x 1 root root 48 2024-09-26 15:32 gcc-rwxr-xr-x 1 root shell 1807376 2024-09-26 15:32 ld-rwxr-xr-x 1 root shell 824400 2024-09-26 15:32 nm-rwxr-xr-x 1 root shell 835784 2024-09-26 15:32 ranlib-rwxr-xr-x 1 root shell 956800 2024-09-26 15:32 strip┌root@localhost:/└─#

The gcc in this GCC toolchain is version gcc 10.2 and the libraries in the GCC toolchain come from the Android API version 21, which are both quite old.

Some of the functions and variables used in current versions of some Unix programs are missing in the libc.so from the API version 21 in the GCC toolchain.

If you get an error like this:

lib/libbison.a(libbison_a-close-stream.o): In function `close_stream':close-stream.c:(.text+0x10): undefined reference to `__fpending'
use aunt Google to check if the missing symbol is defined in the libc.so. If yes, use the tool nm to check if that symbol is defined in the libc.so in the GCC toolchain:

shell@localhost:/data/local/tmp/develop/work/bison-3.8└─$ nm /system/usr/share/gcc/aarch64-linux-android/lib/libc.so | grep __fpending ┌shell@localhost:/data/local/tmp/develop/work/bison-3.8└─$

Instead of checking with Google you can also check the libc of a current Linux version for the PC:

[aarch64-linux-android] [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/ModuleSrc/python_3.10.4 ] $ nm /lib64/libc.so.6 | grep __fpending000000000008c610 T __fpending[aarch64-linux-android] [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/ModuleSrc/python_3.10.4 ] $

Details for the symbol __fpending

fyi: The symbol __fpending from the example above is defined in the libc.so from the API version 23 and newer:
[aarch64-linux-android] [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib ] $ for i in ./aarch64-linux-android/*/libc.so ; do echo "** $i:" ; nm $i | grep __fpending ; done ** ./aarch64-linux-android/21/libc.so: ** ./aarch64-linux-android/22/libc.so: ** ./aarch64-linux-android/23/libc.so: 00000000000180c8 T __fpending ** ./aarch64-linux-android/24/libc.so: 00000000000190c8 T __fpending ** ./aarch64-linux-android/26/libc.so: 000000000001a0c8 T __fpending ** ./aarch64-linux-android/27/libc.so: 000000000001a0c8 T __fpending ** ./aarch64-linux-android/28/libc.so: 000000000001c0c8 T __fpending ** ./aarch64-linux-android/29/libc.so: 000000000001c0c8 T __fpending ** ./aarch64-linux-android/30/libc.so: 000000000001d0c8 T __fpending ** ./aarch64-linux-android/31/libc.so: 000000000001e0c8 T __fpending ** ./aarch64-linux-android/32/libc.so: 000000000001e0c8 T __fpending ** ./aarch64-linux-android/33/libc.so: 000000000001e0c8 T __fpending[aarch64-linux-android] [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/android-ndk-r25.2/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib ] $

If the configure script to create the Makefile for a tool fails with an error message about the missing -fPIE option, the configure script most likely uses gcc to compile and link a test program using only the $CFLAGS. In this case, simply add the parameter for the linker to the CFLAGS, e.g. with

export CFLAGS="-fPIE -pie”

and restart the configure script.

if the linker complains about an "undefined reference to main" while creating a library, remove the option "-pie", example:

┌root@localhost:/data/local/tmp/develop/zlib-1.3.1└─# gcc -shared -Wl,-soname,libz.so.1,--version-script,zlib.map -fPIE -I/data/local/tmp/develop/sysroot/usr/include -fPIC -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN -o libz.so.1.3.1 \> adler32.lo crc32.lo deflate.lo infback.lo inffast.lo inflate.lo inftrees.lo trees.lo zutil.lo compress.lo uncompr.lo gzclose.lo gzlib.lo gzread.lo gzwrite.lo \> -lc -pie -L/data/local/tmp/develop/sysroot/usr/lib/system/usr/share/gcc/bin/../lib/gcc/aarch64-linux-android/10.2.0/../../../../aarch64-linux-android/lib/crtbegin_dynamic.o: In function `_start_main':crtbegin.c:(.text+0x20): undefined reference to `main'crtbegin.c:(.text+0x4c): undefined reference to `main'collect2: error: ld returned 1 exit status┌root@localhost:/data/local/tmp/develop/zlib-1.3.1└─# ┌root@localhost:/data/local/tmp/develop/zlib-1.3.1└─# gcc -shared -Wl,-soname,libz.so.1,--version-script,zlib.map -fPIE -I/data/local/tmp/develop/sysroot/usr/include -fPIC -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN -o libz.so.1.3.1 \> adler32.lo crc32.lo deflate.lo infback.lo inffast.lo inflate.lo inftrees.lo trees.lo zutil.lo compress.lo uncompr.lo gzclose.lo gzlib.lo gzread.lo gzwrite.lo \> -lc -L/data/local/tmp/develop/sysroot/usr/lib┌root@localhost:/data/local/tmp/develop/zlib-1.3.1└─# ┌root@localhost:/data/local/tmp/develop/zlib-1.3.1└─# ls -l libz.so.1.3.1 -rwxr-xr-x 1 root root 159856 2024-10-01 15:13 libz.so.1.3.1┌root@localhost:/data/local/tmp/develop/zlib-1.3.1└─#

Alternativley, you can add the option -nostartfiles and ignore the warning, example:

┌root@localhost:/data/local/tmp/develop/zlib-1.3.1└─# ls -l libz.so.1.3.1 ls: libz.so.1.3.1: No such file or directory┌root@localhost:/data/local/tmp/develop/zlib-1.3.1└─# ┌root@localhost:/data/local/tmp/develop/zlib-1.3.1└─# cc -shared -Wl,-soname,libz.so.1,--version-script,zlib.map -fPIE -I/data/local/tmp/develop/sysroot/usr/include -fPIC -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN -o libz.so.1.3.1 \> adler32.lo crc32.lo deflate.lo infback.lo inffast.lo inflate.lo inftrees.lo trees.lo zutil.lo compress.lo uncompr.lo gzclose.lo gzlib.lo gzread.lo gzwrite.lo \> -lc -pie -L/data/local/tmp/develop/sysroot/usr/lib -nostartfiles/system/usr/share/gcc/bin/../lib/gcc/aarch64-linux-android/10.2.0/../../../../aarch64-linux-android/bin/ld: warning: cannot find entry symbol _start; defaulting to 0000000000000e90┌root@localhost:/data/local/tmp/develop/zlib-1.3.1└─# ┌root@localhost:/data/local/tmp/develop/zlib-1.3.1└─# ls -l libz.so.1.3.1 <-rwxr-xr-x 1 root root 154856 2024-10-01 15:20 libz.so.1.3.1┌root@localhost:/data/local/tmp/develop/zlib-1.3.1└─#

Notes

These instructions were created using the version 1.0.2 of the GCC toolchain.

As of 28.09.2024 I successfully compiled these tools and libraries with the GCC toolchain:

┌root@localhost:/data/local/tmp/develop└─# date Sat Sep 28 11:32:49 CEST 2024┌root@localhost:/data/local/tmp/develop└─# ┌root@localhost:/data/local/tmp/develop└─# ls donebgrep flex-2.6.4 json-c-0.17 libexpat libtasn1-4.19.0 libuv nettle-3.10 pcre2 zstdbrotli-1.0.9 gmp-6.3.0 libconfuse libgdbm libtool-2.5.3 libxml2 nghttp2 pigz-2.8bvi-1.4.2 inetutils-2.5 libevent libiconv-1.17 libunistring-1.2 ncurses-6.3 nmon readline-8.1.2┌root@localhost:/data/local/tmp/develop└─#

A Magisk module with a git binary for Android is available here: https://github.com/henriknelson/git-magisk-module

see also Compiling C programs in Android for how to compile C programs in Termux sessions on the phone.

History of this entry

29.09.2024

initial release

02.10.2024

added infos how to fix the error ""undefined reference to main" while creating a library
added infos about how to create the configure script using autoconf

How to compile Perl for Android

URL: https://xdaforums.com/t/how-to-compile-c-programs-in-android.4683687/#post-89651799

How to compile Perl for Android

Perl for Android can be compiled in a Termux session (see here for general notes about compiling C programs in a Termux session)

Note:

There is already a Perl in the Termux environment but that Perl can not be used in an adb session.

With these instructions I have successfully compiled Perl 5.38 in a Termux session on a ASUS Zenfone 8 with arm64 CPU running OmniROM 14 (Android 14) (root access is required for some of the commands):

First download the Perl source code

wget https://www.cpan.org/src/5.0/perl-5.38.0.tar.gz

Then unpack the tar archive in a Termux session and change the working directory to the directory with the unpacked tar file:

tar -xf perl-5.38.0.tar.gz cd perl-5.38.0

Before starting the configure script for Perl, the default settings for compiling Perl under Android must be corrected in the file hints/linux-android.sh:

Change the value for ldflags to

ldflags="$ldflags -L/system/lib64"Change the value for libpth to
eval "libpth='$libpth /system/lib64 /vendor/libi64'"(or create a new profile in the directory ./hints for Android on 64 Bit CPUs)

To create Perl binaries that can be used in an adb shell, the Perl binaries must be in a directory that the user shell can access. I use the directory

/data/local/tmp/perl538

for that purpose:

su - -c mkdir /data/local/tmp/perl538
su - -c mkdir /data/local/tmp/perl538/bin su - -c mkdir /data/local/tmp/binThese directories can not be accessed by the User for Termux in Android 14 and newer. Therefor temporary change the owner of that directory (these commands must be executed in a Termux session!):
su - -c chown -R $( id -un ):$(id -un ) /data/local/tmp/perl538 su - -c chown $( id -un ):$(id -un ) /data/local/tmp/bin

(The Configure script expects the bin directories to already exist)

Next configure the environment to compile Perl -- either using the default values for compiling Perl without user input:

bash Configure -esd -Alibpth="/system/lib64 /vendor/lib64" -Dprefix=/data/local/tmp/perl538

or by manually entering the configuration values:

bash Configure -es -Alibpth="/system/lib64 /vendor/lib64" -Dprefix=/data/local/tmp/perl538

(Configure is a script in the tar file with the source code for Perl)

If you use the script Configure in interactive mode (without the parameter -d), enter this value for the compiler options to be used:

-fno-strict-aliasing -pipe -fstack-protector-strong -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Wno-implicit-function-declaration -Wno-unused-parameter -Wno-int-conversion -DHAS_GETNET_PROTOS

And enter these libraries to be used:

-lpthread -ldl -lm -lcrypt -lutil -lc -llog -landroid-shmem

For all other question of the Configure script you can confirm the default value.

If you use the script Configure in automatic mode (with the parameter -d ), you must then manually correct some values in the file config.sh created by the script Configure:

ccflags='-fno-strict-aliasing -pipe -fstack-protector-strong -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Wno-implicit-function-declaration -Wno-unused-parameter -Wno-int-conversion -DHAS_GETNET_PROTOS' perllibs='-lpthread -ldl -lm -lcrypt -lutil -lc -llog landroid-shmem'

Then start the build using the command

make

Check the result using

make test

afterwards.

To install Perl into the directory /data/local/tmp/perl538 SELinux must be temporary disabled:

# temporary disable SElinux # su - -c /system/bin/setenforce 0

Then install Perl into the directory /data/local/tmp/perl538 using the command:
make installAfterwards correct the owner for the files in /data/local/tmp/perl538 :
su - -c chown -R shell:shell /data/local/tmp/perl538

and enable SELInux again:
# enable SElinux # su - -c /system/bin/setenforce 1

Trouble Shooting

If make install does not install Perl in the correct directory, check that the directory exists and the Termux user can write to that directory (the directory must already exist when the script Configure is executed). In addition, check the value for the entry installprefix in the file config.sh:
~/perl-5.38.0 $ grep installprefix ./config.sh installprefix='/data/local/tmp/perl538' installprefixexp='.../..' ~/perl-5.38.0 $

All directories with fully qualified path in the file config.sh must already exist:

Use

grep "='/" config.sh

to print these directories.

All directories with three dots in the file config.sh will be created by the make script

Use

grep "\.\.\." config.sh

to print these directories.

Notes

The CFLAG -DHAS_GETNET_PROTOS is probably not neccessary if you answer the questions of the Script Configure correct -- but I do not know the correct answers ...

Compiling Perl 5.40 in Termux fails because the source code for the locale in Perl 5.40 contains new values that are unknown in the current version of Termux

Update 16.08.2024

Instructions to compile Perl 5.40 using the Android SDK can be found in the Examples for compiling C programs for Android using gcc or clang

The Termux version used to compile Perl 5.38 was:

termux-info

~/perl-5.38.0 $ dateThu Aug 8 15:37:35 CEST 2024~/perl-5.38.0 $ ~/perl-5.38.0 $ termux-info Termux Variables: TERMUX_APP_PACKAGE_MANAGER=apt TERMUX_APP__APK_FILE=/data/app/~~CuZfJRR0tYqSFU5MUu5zwQ==/com.termux-fFVK6EOrvGrJxRmu2pEEGQ==/base.apk TERMUX_APP__APK_RELEASE=F_DROID TERMUX_APP__APP_VERSION_CODE=1020 TERMUX_APP__APP_VERSION_NAME=0.119.0-beta.1 TERMUX_APP__DATA_DIR=/data/user/0/com.termux TERMUX_APP__IS_DEBUGGABLE_BUILD=false TERMUX_APP__IS_INSTALLED_ON_EXTERNAL_STORAGE=false TERMUX_APP__PACKAGE_NAME=com.termux TERMUX_APP__PID=4303 TERMUX_APP__TARGET_SDK=28 TERMUX_VERSION=0.119.0-beta.1 TERMUX__SE_FILE_CONTEXT=u:object_r:app_data_file:s0:c140,c256,c512,c768 TERMUX__SE_INFO=default:targetSdkVersion=28:complete TERMUX__SE_PROCESS_CONTEXT=u:r:untrusted_app_27:s0:c140,c256,c512,c768 TERMUX__UID=10140 TERMUX__USER_ID=0 Packages CPU architecture: aarch64 Subscribed repositories: # sources.list deb https://ro.mirror.flokinet.net/termux/termux-main stable main # x11-repo (sources.list.d/x11.list) deb https://ro.mirror.flokinet.net/termux/termux-x11 x11 main # glibc-repo (sources.list.d/glibc.list) deb https://packages-cf.termux.dev/apt/termux-glibc/ glibc stable Updatable packages: apt/stable 2.8.1 aarch64 [upgradable from: 2.7.14-1] command-not-found/stable 2.4.0-36 aarch64 [upgradable from: 2.4.0-34] curl/stable 8.9.1 aarch64 [upgradable from: 8.8.0] git/stable 2.46.0 aarch64 [upgradable from: 2.45.2] gst-plugins-bad/stable 1.24.6 aarch64 [upgradable from: 1.24.5] gst-plugins-base/stable 1.24.6 aarch64 [upgradable from: 1.24.5] gst-plugins-good/stable 1.24.6 aarch64 [upgradable from: 1.24.5] gstreamer/stable 1.24.6 aarch64 [upgradable from: 1.24.5] libc++/stable 27 aarch64 [upgradable from: 26b] libcurl/stable 8.9.1 aarch64 [upgradable from: 8.8.0] libheif/stable 1.18.2 aarch64 [upgradable from: 1.18.1] libnotify/x11 0.8.3-1 aarch64 [upgradable from: 0.8.3] libtirpc/stable 1.3.5 aarch64 [upgradable from: 1.3.4-1] libx11/stable 1.8.10 aarch64 [upgradable from: 1.8.9] libxslt/stable 1.1.42 aarch64 [upgradable from: 1.1.40] libxtst/stable 1.2.5 aarch64 [upgradable from: 1.2.4] ndk-multilib/stable 27 all [upgradable from: 26b] ndk-sysroot/stable 27 aarch64 [upgradable from: 26b-3] openssl/stable 1:3.3.1 aarch64 [upgradable from: 1:3.2.1-1] python-pip/stable 24.2 all [upgradable from: 24.1.2] readline/stable 8.2.13 aarch64 [upgradable from: 8.2.10] termux-tools/stable 1.43.2 all [upgradable from: 1.43.1] vulkan-loader-generic/stable 1.3.292 aarch64 [upgradable from: 1.3.290] termux-tools version: 1.43.1 Android version: 14 Kernel build information: Linux localhost 5.4.147-Omni-qgki-perf-gf532250e9298 #5 SMP PREEMPT Thu Jun 20 18:54:28 CEST 2024 aarch64 Android Device manufacturer: asus Device model: ASUS_I006D LD Variables: LD_LIBRARY_PATH= LD_PRELOAD=/data/data/com.termux/files/usr/lib/libtermux-exec.so ~/perl-5.38.0 $

The config.sh file used to build Perl 5.38 on Android 14 is config.sh_for_perl_5.38_in_Android_14.

The log file of "make test" for the Perl 5.38 executables created is make_test_for_perl538.txt.

For those who only want to use Perl on their phone:

I created a Magisk Module with Perl 5.38 - see the list of Magisk Modules below.

History of this entry

08.08.2024

initial release

Infos for building an Android 14 based Custom ROM image in Q1 2024 or later

URL: https://xdaforums.com/t/infos-for-building-an-android-14-based-customrom-image-in-q1-2024-or-later.4664652/

Infos for building a Custom ROM image in Q1 2024 or later

Notes:

The info below are for building the OmniROM 14 for the ASUS Zenfone 8. But I assume that this is also true for other AOSP based Custom ROMs.
brunch and breakfast are commands defined in the init script for OmniROM and may not be available in other Custom ROMs

Credits

Thanks to the OmniROM developer for this hint.

Attempting to create a new OmniROM image based on Android 14 using the well-known parameter for the command lunch in Q1 2024 or later may fail with this error:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ lunch omni_zenfone8-userInvalid lunch combo: omni_zenfone8-userValid combos must be of the form <product>-<release>-<variant> [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

This is because Google changed the naming conventions for Android releases created with Google's repositories in QPR2 (QPR = Quarterly Platform Releases; QPR2 for Android 14 is the Android Security Patch android-14.0.0_r29 from 05.3.2024) :

It's now necessary to specify a specific version to create a new image using the commands lunch and brunch (see envsetup.sh).

To get the necessary values for the lunch and brunch command check the entries in the file AndroidProducts.mk for your device:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ cat ./device/asus/zenfone8/AndroidProducts.mk PRODUCT_MAKEFILES := \ $(LOCAL_DIR)/omni_zenfone8.mk COMMON_LUNCH_CHOICES := \ omni_zenfone8-ap1a-user \ omni_zenfone8-ap1a-userdebug \ omni_zenfone8-ap1a-eng

The example above is for a ASUS Zenfone 8; the file used for other phones is most probably in a different directory, of course)

So the commands lunch and brunch to build the OmniROM 14 image for an ASUS Zenfone 8 are now:

lunch omni_zenfone8-ap1a-user

brunch zenfone8 ap1a user

(use the parameter eng or userdebug instead of user for one of the other flavours of the ROM image).

The version string used, ap1a, is part of the build_id for the current security patch level:

(see here: https://source.android.com/docs/setup/reference/build-numbers#source-code-tags-and-builds)

To check whether an AOSP source tree uses the new version definition, check whether the repository build/release exists, e.g:

[ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_14.0 ] $ repo info build/release Manifest branch: refs/tags/android-14.0.0_r29 Manifest merge branch: refs/heads/android-14.0 Manifest groups: default,platform-linux ---------------------------- Project: platform/build/release Mount path: /devpool001/develop/OmniROM_14.0/build/release Current revision: b63ef38d4ff1ced7ed8c16733b2b19c8d421aff8 Manifest revision: refs/tags/android-14.0.0_r29 Local Branches: 0 ---------------------------- [ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_14.0 ] $

In an AOSP source tree using the old version definitions the output looks like this:

[ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_14.0_patch_2024-02-05 ] $ repo info build/release Manifest branch: refs/tags/android-14.0.0_r27 Manifest merge branch: refs/heads/android-14.0 Manifest groups: default,platform-linux ---------------------------- error: project build/release not found[ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_14.0_patch_2024-02-05 ] $

I wrote a little script, print_security_patch, to print the current Security Patch level and build id for an AOSP based repository tree.
This script works for repositories that were last updated before Q1 2024 and after Q1 2024.

The script was only tested with the repositories for OmniROM 14 but I think it should work with every AOSP based repository tree.

Usage:
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ /data/develop/android/scripts_on_linux/print_security_patch -h print_security_patch - print the security patch and the the build id used in an repository tree for AOSP This script works for repositories that were last updated before Q1 2024 and after Q1 2024. The script must be executed at the top level in a repository tree for AOSP [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

Examples

executed in the top level of an Android repository tree using the old definitions

[ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_14.0_patch_2024-02-05 ] $ /data/develop/android/scripts_on_linux/print_security_patch ; echo "RC=$?" The repository uses the old definitions The current security patch is "2024-02-05" The current build_id is "UP1A" RC=1 [ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_14.0_patch_2024-02-05 ]

executed in the top level of an Android repository tree using the new definitions

[ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_14.0 ] $ /data/develop/android/scripts_on_linux/print_security_patch ; echo "RC=$?" The repository uses the new definitions The current security patch is "2024-03-05" The current build_id is "ap1a" RC=0 [ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_14.0 ] $

sourced in in the top level of an Android repository tree using the new definitions

[ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_14.0 ] $ . /data/develop/android/scripts_on_linux/print_security_patch The repository uses the new definitions The current security patch is "2024-03-05" The current build_id is "ap1a" [ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_14.0 ] $ echo $RELEASE_PLATFORM_SECURITY_PATCH 2024-03-05 [ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_14.0 ] $ echo $BUILD_ID ap1a [ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_14.0 ] $ echo $REPO_FORMAT new [ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_14.0 ] $

The script can be downloaded from here: print_security_patch

Update 09.04.2024/bs

There is now a new version of the script available - see How to print the security patch level used in repositories or ROM image zip files for details.

Notes

Note that as of 29.03.2024 the command breakfast does not work anymore in repository trees with the new version definitions.

See How to compile the OmniROM 14 for the the ASUS Zenfone 8 for general instructions how to create your own Custom ROM image with OmniROM 14.

How to sign a zip file with a Custom ROM image

URL: not yet published

How to sign a zip file with a Custom ROM image

To sign a ZIP file use the tool apksigner.jar from the SDK

E.g.

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ java -jar ../Sdk/build-tools/34.0.0/lib/apksigner.jar sign --cert ../OmniROM_14.0/build/make/target/product/security/platform.x509.pem --key ../OmniROM_14.0/build/make/target/product/security/platform.pk8 --min-sdk-version 34 out/target/product/zenfone8/omni-14-20240401-zenfone8-MICROG.zip [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $The SDK is part of the repositories used to build a Custom ROM image. It is also available for download here: https://developer.android.com/studio/releases/platform-tools

To verify the certificate of a ZIP file a tool like update_verifier provided by LineageOS can be used. update_verifier is a python script and available for download here: https://wiki.lineageos.org/verifying-builds

To install the tool update_verifier do:

# install the tool # [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $ git clone https://github.com/LineageOS/update_verifier Cloning into 'update_verifier'... remote: Enumerating objects: 127, done. remote: Counting objects: 100% (3/3), done. remote: Compressing objects: 100% (2/2), done. remote: Total 127 (delta 0), reused 2 (delta 0), pack-reused 124 Receiving objects: 100% (127/127), 36.40 KiB | 36.40 MiB/s, done. Resolving deltas: 100% (52/52), done. # install required php files # [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/update_verifier ] $ pip3 install -r requirements.txt Defaulting to user installation because normal site-packages is not writeable Collecting oscrypto==1.3.0 (from -r requirements.txt (line 1)) Obtaining dependency information for oscrypto==1.3.0 from https://files.pythonhosted.org/packages/01/7c/fa07d3da2b6253eb8474be16eab2eadf670460e364ccc895ca7ff388ee30/oscrypto-1.3.0-py2.py3-none-any.whl.metadata Downloading oscrypto-1.3.0-py2.py3-none-any.whl.metadata (15 kB) Requirement already satisfied: asn1crypto==1.5.1 in /usr/lib/python3.12/site-packages (from -r requirements.txt (line 2)) (1.5.1) Downloading oscrypto-1.3.0-py2.py3-none-any.whl (194 kB) â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â” 194.6/194.6 kB 2.9 MB/s eta 0:00:00 Installing collected packages: oscrypto Successfully installed osrpyto-1.3.0

update_verifier needs the public key or the certificate used to sign the ZIP file; the public key for the LineageOS image files is part of the repository for the tool so checking a LineageOS image file works out of the box:

# check the certificate of an LineageOS ROM image file: # [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/update_verifier ] $python3 update_verifier.py lineageos_pubkey /data/backup/ASUS_ZENFONE8/Lineage-20/lineage-20.0-20230526-nightly-sake-signed.zip verified successfully [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/update_verifier ] $

To test the certificate for ZIP files for other ROMs the certificate in the the ZIP file can be used; the certificate in the ZIP file is the file META-INF/com/android/otacert in the ZIP file:
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/update_verifier ] $unzip -p /data/backup/ASUS_ZENFONE8/omnirom_local/omni-14/omni-14-20240407-zenfone8-MICROG.zip META-INF/com/android/otacert >otacert [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/update_verifier ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/update_verifier ] $ ls -l otacert -rw-rw-r--. 1 xtrnaw7 xtrnaw7 1675 Apr 8 15:48 otacert [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/update_verifier ] $

Now test the certificate of the ZIP file:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/update_verifier ] $ python3 update_verifier.py otacert /data/backup/ASUS_ZENFONE8/omnirom_local/omni-14/omni-14-20240407-zenfone8-MICROG.zip verified successfully [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/update_verifier ] $

To print the contents of the certificate use the openssl command, e.g.

openssl x509 -in testkey.x509.pem -text

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/update_verifier ] $ openssl x509 -in testkey.x509.pem -text Certificate: Data: Version: 3 (0x2) Serial Number: 93:6e:ac:be:07:f2:01:df Signature Algorithm: sha1WithRSAEncryption Issuer: C = US, ST = California, L = Mountain View, O = Android, OU = Android, CN = Android, emailAddress = android@android.com Validity Not Before: Feb 29 01:33:46 2008 GMT Not After : Jul 17 01:33:46 2035 GMT Subject: C = US, ST = California, L = Mountain View, O = Android, OU = Android, CN = Android, emailAddress = android@android.com Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) Modulus: 00:d6:93:19:04:de:c6:0b:24:b1:ed:c7:62:e0:d9: d8:25:3e:3e:cd:6c:eb:1d:e2:ff:06:8c:a8:e8:bc: a8:cd:6b:d3:78:6e:a7:0a:a7:6c:e6:0e:bb:0f:99: 35:59:ff:d9:3e:77:a9:43:e7:e8:3d:4b:64:b8:e4: fe:a2:d3:e6:56:f1:e2:67:a8:1b:bf:b2:30:b5:78: c2:04:43:be:4c:72:18:b8:46:f5:21:15:86:f0:38: a1:4e:89:c2:be:38:7f:8e:be:cf:8f:ca:c3:da:1e: e3:30:c9:ea:93:d0:a7:c3:dc:4a:f3:50:22:0d:50: 08:07:32:e0:80:97:17:ee:6a:05:33:59:e6:a6:94: ec:2c:b3:f2:84:a0:a4:66:c8:7a:94:d8:3b:31:09: 3a:67:37:2e:2f:64:12:c0:6e:6d:42:f1:58:18:df: fe:03:81:cc:0c:d4:44:da:6c:dd:c3:b8:24:58:19: 48:01:b3:25:64:13:4f:bf:de:98:c9:28:77:48:db: f5:67:6a:54:0d:81:54:c8:bb:ca:07:b9:e2:47:55: 33:11:c4:6b:9a:f7:6f:de:ec:cc:8e:69:e7:c8:a2: d0:8e:78:26:20:94:3f:99:72:7d:3c:04:fe:72:99: 1d:99:df:9b:ae:38:a0:b2:17:7f:a3:1d:5b:6a:fe: e9:1f Exponent: 3 (0x3) X509v3 extensions: X509v3 Subject Key Identifier: 48:59:00:56:3D:27:2C:46:AE:11:86:05:A4:74:19:AC:09:CA:8C:11 X509v3 Authority Key Identifier: keyid:48:59:00:56:3D:27:2C:46:AE:11:86:05:A4:74:19:AC:09:CA:8C:11 DirName:/C=US/ST=California/L=Mountain View/O=Android/OU=Android/CN=Android/emailAddress=android@android.com serial:93:6E:AC:BE:07:F2:01:DF X509v3 Basic Constraints: CA:TRUE Signature Algorithm: sha1WithRSAEncryption Signature Value: 7a:af:96:8c:eb:50:c4:41:05:51:18:d0:da:ab:af:01:5b:8a: 76:5a:27:a7:15:a2:c2:b4:4f:22:14:15:ff:da:ce:03:09:5a: bf:a4:2d:f7:07:08:72:6c:20:69:e5:c3:6e:dd:ae:04:00:be: 29:45:2c:08:4b:c2:7e:b6:a1:7e:ac:9d:be:18:2c:20:4e:b1: 53:11:f4:55:d8:24:b6:56:db:e4:dc:22:40:91:2d:75:86:fe: 88:95:1d:01:a8:fe:b5:ae:5a:42:60:53:5d:f8:34:31:05:24: 22:46:8c:36:e2:2c:2a:5e:f9:94:d6:1d:d7:30:6a:e4:c9:f6: 95:1b:a3:c1:2f:1d:19:14:dd:c6:1f:1a:62:da:2d:f8:27:f6: 03:fe:a5:60:3b:2c:54:0d:bd:7c:01:9c:36:ba:b2:9a:42:71: c1:17:df:52:3c:db:c5:f3:81:7a:49:e0:ef:a6:0c:bd:7f:74: 17:7e:7a:4f:19:3d:43:f4:22:07:72:66:6e:4c:4d:83:e1:bd: 5a:86:08:7c:f3:4f:2d:ec:21:e2:45:ca:6c:2b:b0:16:e6:83: 63:80:50:d2:c4:30:ee:a7:c2:6a:1c:49:d3:76:0a:58:ab:7f: 1a:82:cc:93:8b:48:31:38:43:24:bd:04:01:fa:12:16:3a:50: 57:0e:68:4d -----BEGIN CERTIFICATE----- MIIEqDCCA5CgAwIBAgIJAJNurL4H8gHfMA0GCSqGSIb3DQEBBQUAMIGUMQswCQYD VQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTEWMBQGA1UEBxMNTW91bnRhaW4g VmlldzEQMA4GA1UEChMHQW5kcm9pZDEQMA4GA1UECxMHQW5kcm9pZDEQMA4GA1UE AxMHQW5kcm9pZDEiMCAGCSqGSIb3DQEJARYTYW5kcm9pZEBhbmRyb2lkLmNvbTAe Fw0wODAyMjkwMTMzNDZaFw0zNTA3MTcwMTMzNDZaMIGUMQswCQYDVQQGEwJVUzET MBEGA1UECBMKQ2FsaWZvcm5pYTEWMBQGA1UEBxMNTW91bnRhaW4gVmlldzEQMA4G A1UEChMHQW5kcm9pZDEQMA4GA1UECxMHQW5kcm9pZDEQMA4GA1UEAxMHQW5kcm9p ZDEiMCAGCSqGSIb3DQEJARYTYW5kcm9pZEBhbmRyb2lkLmNvbTCCASAwDQYJKoZI hvcNAQEBBQADggENADCCAQgCggEBANaTGQTexgskse3HYuDZ2CU+Ps1s6x3i/waM qOi8qM1r03hupwqnbOYOuw+ZNVn/2T53qUPn6D1LZLjk/qLT5lbx4meoG7+yMLV4 wgRDvkxyGLhG9SEVhvA4oU6Jwr44f46+z4/Kw9oe4zDJ6pPQp8PcSvNQIg1QCAcy 4ICXF+5qBTNZ5qaU7Cyz8oSgpGbIepTYOzEJOmc3Li9kEsBubULxWBjf/gOBzAzU RNps3cO4JFgZSAGzJWQTT7/emMkod0jb9WdqVA2BVMi7yge54kdVMxHEa5r3b97s zI5p58ii0I54JiCUP5lyfTwE/nKZHZnfm644oLIXf6MdW2r+6R8CAQOjgfwwgfkw HQYDVR0OBBYEFEhZAFY9JyxGrhGGBaR0GawJyowRMIHJBgNVHSMEgcEwgb6AFEhZ AFY9JyxGrhGGBaR0GawJyowRoYGapIGXMIGUMQswCQYDVQQGEwJVUzETMBEGA1UE CBMKQ2FsaWZvcm5pYTEWMBQGA1UEBxMNTW91bnRhaW4gVmlldzEQMA4GA1UEChMH QW5kcm9pZDEQMA4GA1UECxMHQW5kcm9pZDEQMA4GA1UEAxMHQW5kcm9pZDEiMCAG CSqGSIb3DQEJARYTYW5kcm9pZEBhbmRyb2lkLmNvbYIJAJNurL4H8gHfMAwGA1Ud EwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADggEBAHqvlozrUMRBBVEY0NqrrwFbinZa J6cVosK0TyIUFf/azgMJWr+kLfcHCHJsIGnlw27drgQAvilFLAhLwn62oX6snb4Y LCBOsVMR9FXYJLZW2+TcIkCRLXWG/oiVHQGo/rWuWkJgU134NDEFJCJGjDbiLCpe +ZTWHdcwauTJ9pUbo8EvHRkU3cYfGmLaLfgn9gP+pWA7LFQNvXwBnDa6sppCccEX 31I828XzgXpJ4O+mDL1/dBd+ek8ZPUP0IgdyZm5MTYPhvVqGCHzzTy3sIeJFymwr sBbmg2OAUNLEMO6nwmocSdN2ClirfxqCzJOLSDE4QyS9BAH6EhY6UFcOaE0= -----END CERTIFICATE----- [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/update_verifier ] $

Notes

See the section How to compile the OmniROM 14 for the the ASUS Zenfone 8 for how to create the certificates

See https://source.android.com/docs/core/ota/sign_builds for details

The certificate for other ROMs is also part of the running OS; the ZIP file with the certificate is always /system/etc/security/otacerts.zip.

So to download the certificate from a running OS do:

# download the ZIP file with the public key from the OS # [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/update_verifier ] $ adb pull /system/etc/security/otacerts.zip /system/etc/security/otacerts.zip: 1 file pulled, 0 skipped. 0.3 MB/s (1089 bytes in 0.004s) [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/update_verifier ] $ # unpack the zip file with the public key # [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/update_verifier ] $ unzip otacerts.zip Archive: otacerts.zip inflating: testkey.x509.pem [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/update_verifier ] $

There is only one file in the ZIP file and that is the public key used (the filename may be different).

For those not familar with ceritficates and public / private keys: A public key can only be used to validate the ZIP file - to sign the ZIP file the private key is required. And the private key should never be available for the public.

apksigner.jar can also be used so sign an apk file, e.g.:

# check the sign of the apk file # [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ java -jar ../Sdk/build-tools/34.0.0/lib/apksigner.jar verify ../AsusFMRadio/AsusFMService.apk DOES NOT VERIFY ERROR: No JAR signatures [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ # sign the apk file # [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ java -jar ../Sdk/build-tools/34.0.0/lib/apksigner.jar sign --cert ../OmniROM_14.0/build/make/target/product/security/platform.x509.pem --key ../OmniROM_14.0/build/make/target/product/security/platform.pk8 --min-sdk-version 34 ../AsusFMRadio/AsusFMService.apk [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ # check the result # [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ java -jar ../Sdk/build-tools/34.0.0/lib/apksigner.jar verify --verbose ../AsusFMRadio/AsusFMService.apk Verifies Verified using v1 scheme (JAR signing): false Verified using v2 scheme (APK Signature Scheme v2): false Verified using v3 scheme (APK Signature Scheme v3): true Verified using v3.1 scheme (APK Signature Scheme v3.1): false Verified using v4 scheme (APK Signature Scheme v4): false Verified for SourceStamp: false Number of signers: 1 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

----

How to get the security patch level in a ROM image zip file

URL: https://xdaforums.com/t/how-to-get-the-security-patch-level-used-in-a-rom-image-zip-file.4666081/

How to get the security patch level in a ROM image zip file

Sometimes it is useful to check the security patch level that was used to create a ROM image ZIP file for an Android phone (e.g. to decide whether an OS update is required).

To check the security patch level used in an ZIP file with a ROM image for an Android phone check the contents of the file META-INF/com/android/metadata in the ZIP file, the security patch level is the value for the key post-security-patch-level.

Example:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/omnirom/omni-14 ] $ unzip -p omni-14-202404041405-zenfone8-MICROG.zip META-INF/com/android/metadata ota-property-files=payload_metadata.bin:3854:100364,payload.bin:3854:1368231516,payload_properties.txt:1368235428:156,apex_info.pb:2060:1079,care_map.pb:3186:621,metadata:69:668,metadata.pb:805:1207 ota-required-cache=0 ota-streaming-property-files=payload.bin:3854:1368231516,payload_properties.txt:1368235428:156,apex_info.pb:2060:1079,care_map.pb:3186:621,metadata:69:668,metadata.pb:805:1207 ota-type=AB post-build=asus/WW_I006D/ASUS_I006D:14/AP1A.240305.019.A1/289:user/release-keys post-build-incremental=289 post-sdk-level=34post-security-patch-level=2024-03-05post-timestamp=1712239453 pre-device=ASUS_I006D [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/omnirom/omni-14 ] $ # or to print only the security patch level [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/omnirom/omni-14 ] $ unzip -p omni-14-202404041405-zenfone8-MICROG.zip META-INF/com/android/metadata | grep "^post-security-patch-level=" | cut -f2 -d "=" 2024-03-05 [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/omnirom/omni-14 ] $

Other examples:

ASUS Android 13 for the ASUS Zenfone 8

# ASUS Android 13 for the ASUS Zenfone 8 # [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/ASUS_firmware/Android_13 ] $ unzip -p UL-ASUS_I006D-ASUS-33.0210.0210.269-1.1.26-2303-user.zip META-INF/com/android/metadata ota-property-files=payload_metadata.bin:3634:227939,payload.bin:3634:3395083562,payload_properties.txt:3395087254:156,apex_info.pb:2030:839,care_map.pb:2916:671,metadata:69:689,metadata.pb:826:1156 ota-required-cache=0 ota-streaming-property-files=payload.bin:3634:3395083562,payload_properties.txt:3395087254:156,apex_info.pb:2030:839,care_map.pb:2916:671,metadata:69:689,metadata.pb:826:1156 ota-type=AB post-build=asus/WW_I006D/ASUS_I006D:13/TKQ1.220807.001/33.0210.0210.269:user/release-keys post-build-incremental=33.0210.0210.269 post-sdk-level=33 post-security-patch-level=2023-03-05post-timestamp=1678292389 pre-device=ASUS_I006D [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/ASUS_firmware/Android_13 ] $

LineageOS 21 for the ASUS Zenfone 8

# LineageOS 21 for the ASUS Zenfone 8 # [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/Lineage-21 ] $ unzip -p lineage-21.0-20231113-UNOFFICIAL-sake.zip META-INF/com/android/metadata ota-property-files=payload_metadata.bin:4276:114414,payload.bin:4276:1128628633,payload_properties.txt:1128632967:156,apex_info.pb:2281:1101,care_map.pb:3429:800,metadata:69:701,metadata.pb:838:1395 ota-required-cache=0 ota-streaming-property-files=payload.bin:4276:1128628633,payload_properties.txt:1128632967:156,apex_info.pb:2281:1101,care_map.pb:3429:800,metadata:69:701,metadata.pb:838:1395 ota-type=AB post-build=asus/WW_I006D/ASUS_I006D:13/TKQ1.220807.001/33.0210.0210.296:user/release-keys post-build-incremental=eng.mikooo.20231110.195404 post-sdk-level=34 post-security-patch-level=2023-11-01post-timestamp=1699663738 pre-device=ASUS_I006D [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/Lineage-21 ] $

(see also How to print the security patch level used in repositories or ROM image zip files)

To print the security patch level of an installed and running Android OS check the value of the property ro.build.version.security_patch, e.g.:

ASUS_I006D:/ $ getprop ro.build.version.security_patch 2024-04-05 ASUS_I006D:/ $

Another important info in the file META-INF/com/android/metadata in the ZIP file is the value for the key post-timestamp .

In the example above, this is this line

post-timestamp=1712239453 To print the timestamps used to create the Android OS installed and running on the phone check the values of these properties:

ASUS_I006D:/ $ getprop | grep build.date | sed -e "/utc/ s/$/\n/g" [ro.bootimage.build.date]: [Thu Apr 4 07:49:54 CEST 2024] [ro.bootimage.build.date.utc]: [1712209794] [ro.build.date]: [Thu Apr 4 07:49:54 CEST 2024] [ro.build.date.utc]: [1712209794] [ro.odm.build.date]: [Thu Apr 4 07:49:54 CEST 2024] [ro.odm.build.date.utc]: [1712209794] [ro.product.build.date]: [Thu Apr 4 07:49:54 CEST 2024] [ro.product.build.date.utc]: [1712209794] [ro.system.build.date]: [Thu Apr 4 07:49:54 CEST 2024] [ro.system.build.date.utc]: [1712209794] [ro.system_ext.build.date]: [Thu Apr 4 07:49:54 CEST 2024] [ro.system_ext.build.date.utc]: [1712209794] [ro.vendor.build.date]: [Thu Apr 4 07:49:54 CEST 2024] [ro.vendor.build.date.utc]: [1712209794] [ro.vendor_dlkm.build.date]: [Thu Apr 4 07:49:54 CEST 2024] [ro.vendor_dlkm.build.date.utc]: [1712209794]Since some time now the updater from Android refuses to downgrade the installed OS -- the attempt to install an operating system image with an older timestamp than the timestamp of the installed operating system fails with this error:

Updated failed with error 51

and a message like this will appear in the logcat:

ASUS_I006D:/ $ logcat | grep timestamp 04-07 18:24:29.600 4348 4365 D OmniOta : latestDeviceBuild = BuildImage(filename=omni-14-202404041405-zenfone8-MICROG.zip, timestamp=1712243815, size=1368238540) 04-07 18:24:40.796 1467 1467 E update_engine: [ERROR:delta_performer.cc(1161)] The current OS build timestamp (1712209794) is newer than the maximum timestamp in the manifest (1711954765) ^C

Note:

1712209794 is the number of seconds since 1.1.1970. To convert the date into an human readable date string use this command:

[ OmniRom 14 Dev - xtrnaw7@t15g / ] $ date --date='@1711954765' Mon Apr 1 08:59:25 AM CEST 2024 [ OmniRom 14 Dev - xtrnaw7@t15g / ] $

or, to print the date in UTC:

[ OmniRom 14 Dev - xtrnaw7@t15g / ] $ date -u --date='@1711954765' Mon Apr 1 06:59:25 AM UTC 2024 [ OmniRom 14 Dev - xtrnaw7@t15g / ] $
Be aware that the value for the timestamp in the ZIP file is independent from the timestamp of the ZIP file; it is also not related to the time the image was created.

History of this entry

08.04.2024 /bs

initial release

----

How to print the security patch level used in repositories or ROM image zip files

URL: https://xdaforums.com/t/how-to-print-the-security-patch-level-used-in-repositories-or-rom-image-zip-files.4666170/

How to print the security patch level used in repositories or ROM image zip files

For those who work with multiple repositories for custom ROMs and/or multiple ROM zip files for Android phones, it might be useful to determine the security level used in the repositories and/or zip files with a simple command

I have written a small shell script for Linux that can do this:

print_security_patch

The usage for the script is:

[ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ print_security_patch -h print_security_patch - print either the security patch of a repository tree for AOSP or an Android ROM zip file This script works for repositories that were last updated before and after the release of Android 14 QPR2 (in Q1 2024) If the script is called without parameters, it must be executed at the top level in a repository tree for AOSP Usage: print_security_patch [rom_zip_file|repository_tree] [...] [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $

Usage examples

- Print the security patch level of a repository for a custom ROM for Android 14 QPR2 or newer in the current directory:
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ print_security_patch Retrieving the security patch level for the repository at "/data/develop/android/OmniROM_14.0" ... The repository uses the new definitionsThe current security patch is "2024-03-05" The current build_id is "ap1a" [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ or in another output format: [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ print_security_patch $PWD Repository: /data/develop/android/OmniROM_14.0, Patch Level: 2024-03-05, Build ID: ap1a, Repository format: new [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

- Print the security patch level of a repository for a Custom ROM for Android 14 without QPR2 or Android 13 or older in the current directory:

[ OmniRom 13 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_13.0 ] $ print_security_patch Retrieving the security patch level for the repository at "/devpool001/develop/OmniROM_13.0" ... The repository uses the old definitionsThe current security patch is "2023-09-01" The current build_id is "TP1A" [ OmniRom 13 Dev - xtrnaw7@t15g /devpool001/develop/OmniROM_13.0 ] $

- Print the security patch level of a repository for a custom ROM in the current directory and store the values in environment variables :

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ . print_security_patch Retrieving the security patch level for the repository at "/data/develop/android/OmniROM_14.0" ... The repository uses the new definitions The current security patch is "2024-03-05" The current build_id is "ap1a" [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ echo $RELEASE_PLATFORM_SECURITY_PATCH 2024-03-05 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ echo $BUILD_ID ap1a [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ echo $REPO_FORMAT new [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

This usage is quite useful in a script to build a new ROM image for Android 14 with QPR2 and newer that works also after the BUILD_ID changed in a repository (e.g after applying a patch):

. build/envsetup.sh . ../scripts/print_security_patch if [ "${REPO_FORMAT}"x = "new"x ] ; then LUNCH_PARAMETER="omni_zenfone8-${BUILD_ID}-user" BRUNCH_PARAMETER="zenfone8 ${BUILD_ID} user" lunch ${LUNCH_PARAMETER} brunch ${BRUNCH_PARAMETER}
fi

- Print the security patch level for multiple repositories for custom ROMs:

[ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ print_security_patch /devpool001/develop/OmniROM_1* Repository: /devpool001/develop/OmniROM_13.0, Patch Level: 2023-09-01, Build ID: TP1A, Repository format: old Repository: /devpool001/develop/OmniROM_14.0, Patch Level: 2024-04-05, Build ID: ap1a, Repository format: new Repository: /devpool001/develop/OmniROM_14.0_new, Patch Level: 2024-03-05, Build ID: ap1a, Repository format: new Repository: /devpool001/develop/OmniROM_14.0_patch_2024-02-05, Patch Level: 2024-02-05, Build ID: UP1A, Repository format: old [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $

- Print the security patch level used to build a ROM image file:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ print_security_patch out/target/product/zenfone8/omni-14-20240401-zenfone8-MICROG.zip ZIP File: out/target/product/zenfone8/omni-14-20240401-zenfone8-MICROG.zip, Patch Level: 2024-03-05 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

- Print the security patch level for multiple ROM image files:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ print_security_patch /data/backup/ASUS_ZENFONE8/Lineage-*/*zip /data/backup/ASUS_ZENFONE8/omnirom/*/*zip /data/backup/ASUS_ZENFONE8/omnirom_local/*/*zip ZIP File: /data/backup/ASUS_ZENFONE8/Lineage-18_Android11/lineage-18.1-20220311-nightly-sake-signed.zip, Patch Level: 2022-02-05 ZIP File: /data/backup/ASUS_ZENFONE8/Lineage-19_Android12/lineage-19.1-20220923-nightly-sake-signed.zip, Patch Level: 2022-09-05 ZIP File: /data/backup/ASUS_ZENFONE8/Lineage-19_Android12/lineage-19.1-20221216-nightly-sake-signed.zip, Patch Level: 2022-12-05 ZIP File: /data/backup/ASUS_ZENFONE8/Lineage-19_Android12_with_MicroG/lineage-19.1-20221222-microG-sake.zip, Patch Level: 2022-12-05 ZIP File: /data/backup/ASUS_ZENFONE8/Lineage-20/lineage-20.0-20230526-nightly-sake-signed.zip, Patch Level: 2023-05-05 ZIP File: /data/backup/ASUS_ZENFONE8/Lineage-21/lineage-21.0-20231113-UNOFFICIAL-sake.zip, Patch Level: 2023-11-01 ZIP File: /data/backup/ASUS_ZENFONE8/omnirom/omni-12/omni-12-20220807-zenfone8-MICROG.zip, Patch Level: 2022-05-05 ZIP File: /data/backup/ASUS_ZENFONE8/omnirom/omni-12/omni-12-20220814-zenfone8-MICROG.zip, Patch Level: 2022-05-05 ZIP File: /data/backup/ASUS_ZENFONE8/omnirom/omni-13/omni-13-202307291409-zenfone8-MICROG.zip, Patch Level: 2023-07-05 ZIP File: /data/backup/ASUS_ZENFONE8/omnirom/omni-13/omni-13-202307291412-zenfone8-GAPPS.zip, Patch Level: 2023-07-05 ZIP File: /data/backup/ASUS_ZENFONE8/omnirom/omni-13/omni-13-202308121832-zenfone8-MICROG.zip, Patch Level: 2023-08-05 ZIP File: /data/backup/ASUS_ZENFONE8/omnirom/omni-13/omni-13-202308130136-zenfone8-WEEKLY.zip, Patch Level: 2023-08-05 ZIP File: /data/backup/ASUS_ZENFONE8/omnirom/omni-14/omni-14-202404041405-zenfone8-MICROG.zip, Patch Level: 2024-03-05 ZIP File: /data/backup/ASUS_ZENFONE8/omnirom_local/omni-12/omni-12-20220927-zenfone8-MICROG.zip, Patch Level: 2022-05-05 ZIP File: /data/backup/ASUS_ZENFONE8/omnirom_local/omni-13/omni-13-202309302115-zenfone8-WEEKLY.zip, Patch Level: 2023-09-01 ZIP File: /data/backup/ASUS_ZENFONE8/omnirom_local/omni-13/omni-13-20231203-zenfone8-MICROG.zip, Patch Level: 2023-09-01 ZIP File: /data/backup/ASUS_ZENFONE8/omnirom_local/omni-14/omni-14-20240408-zenfone8-MICROG.zip, Patch Level: 2024-04-05 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

The script writes all error messages to STDERR; to ignore error messages use

print_security_patch ... 2>/dev/null

The script can be downloaded from here: print_security_patch

See the How to get the security patch level in a ROM image zip file and Infos for building an Android 14 based Custom ROM image in Q1 2024 or later for details how to retrieve the infos about the security level.

How to fix a hanging restore in Titanium Backup

URL: https://xdaforums.com/t/how-to-fix-a-hanging-restore-in-titanium-backup.4666620/

How to fix a hanging restore in Titanium Backup

IMHO Titanium Backup is still one of the best apps to backup and restore apps and data in the Android OS.

Update 08.07.2024

Unfortunately, this is no longer the case - since Android uses different users and permissions for each app, restoring an app via Titanium Backup is only possible with manual intervention.

Nevertheless, the information below is still valid

But sometimes the Titanium Backup just hangs while restoring files.

In this case the first checks should be (as already mentioned in various other posts, for example here: https://xdaforums.com/t/tibu-stuck-at-0-restoring-apps-anyone-else.3569959/#post-71347448 ):

"First step go into settings/developer options and look for "verify apps over USB" and tick that off.

Now go into titanium backup go into menu/preferences scroll to the very bottom to troubleshooting settings then select app processing mode and then select auto/indirect. Now reboot and Titanium backup will restore your apps and data in batches or individually just like before! "

If this does not fix the problem and you're running Android 14 or newer, the error could be caused by you trying to restore a somehow "outdated" app. Titanium Backup creates apk files in it's data directory to be able to restore the apps later. So to check this, open an adb shell and try to install the apk file from the backup manual using the command "pm install", e.g.: ", e.g.:

ASUS_I006D:/ $ cd /sdcard/TitaniumBackupASUS_I006D:/sdcard/TitaniumBackup $ ASUS_I006D:/sdcard/TitaniumBackup $ cat net.momodalo.app.vimtouch-5d7d195f2c9f5a0d04facb2dd060f73f.apk | pm install -S 3611858 Failure [INSTALL_FAILED_DEPRECATED_SDK_VERSION: App package must target at least SDK version 23, but found 17] 1|ASUS_I006D:/sdcard/TitaniumBackup $

Android 14 does not allow the installation of apps with outdated target SDK version anymore:

"Starting with Android 14, apps with a targetSdkVersion lower than 23 can't be installed. Requiring apps to meet these minimum target API level requirements improves security and privacy for users."

see https://developer.android.com/about/versions/14/behavior-changes-all

AFAIK the only method to install packages with an outdated target SDK version is manuall in an adb shell with the additional parameter --bypass-low-target-sdk-block for the command "pm install", example:

ASUS_I006D:/sdcard/TitaniumBackup $ cat net.momodalo.app.vimtouch-5d7d195f2c9f5a0d04facb2dd060f73f.apk | pm install --bypass-low-target-sdk-block -S 3611858Success ASUS_I006D:/sdcard/TitaniumBackup $

Unfortunately Titanium Backup does not support this approach and therefor just hangs trying to install an outdated App (and I don't think there will be a new, fixed version from Titanum Backup in the future ...)

Therefore, to continue the restore via Titanium Backup, either delete all apk files with outdated target sdk version from the directory /sdcard/TitaniumBackup on the phone and then execute Titanium Backup again or manually reinstall these apps before running Titanium Backup. Deleting the apk files is not really a sensible method here; the better solution is to install all the apps involved manually before starting Titanium Backup.

Since it is a bit difficult to find out whether an apk file is affected by this problem or not and there may be a lot of apks with this issue, I have created a small script for it:

check_apks.sh

The usage for check_apks.sh is

[ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ check_apks.sh -h check_apks.sh - check the target sdk version of apk files Usage:    cd <dir_with_apk_files>    check_apks.sh [sdk=nn] [for_android] > <output_file_for_the_pm_commands_to_install_the_apks> The parameter "sdk=nnn" can be used to change the SDK version used to check the target sdk version in the apk files If the parameter "for_android" is used, the script creates commands that must be executed in a shell on the phone even if running on a PC. Use the environment variable AAPT to define the aapt executable to be used (if not available via PATH variable) Use the environment variable TMPFILE_DIR to define a different directory for temporary files The default sdk version for the check is 23 The default directory for temporary files is /tmp The aapt executable to use is "/data/develop/android/android_sdk/build-tools/34.0.0/aapt" [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $

The script can be executed on a PC running Linux or on the phone running the Android OS. The only requirement is the executable aapt from the Android SDK.

The Android SDK can be downloaded from here https://developer.android.com/tools; the executable aapt is also part of the repositories to build a custom ROM for Android (at least in the repositories for the OmniROM; for Linux the executable is ./prebuilts/sdk/tools/linux/bin/aapt).

check_apks.sh checks the configured target SDK version in the apk file for each apk file in the current directory and, if this is too low, writes the necessary pm install command for manual installation of the apk file to STDOUT.

for example (running check_apks.sh on a PC with a copy of the directory /sdcard/TitaniumBckup from the phone):

OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/data_backup/ASUS_I006D/TitaniumBackup ] $ check_apks.sh check_apks.sh - check the target sdk version of apk files NOT running in the Android OS Creating "pm install" commands that must be executed on the PC Searching for apk files with a target sdk value less then 23 The pm commands to install the apks will be written to "standard output (use redirection for stdout (> <filename>) to write to a file)" Processing the apk files in the directory "/data/backup/ASUS_ZENFONE8/data_backup/ASUS_I006D/TitaniumBackup" ... 3 apk(s) found Creating the list of latest apk files ..: ... 3 unique apk(s) found Processing the file "net.momodalo.app.vimtouch-5d7d195f2c9f5a0d04facb2dd060f73f.apk" ...     This is the app "Vi IMproved Touch 2.3"; the target sdk version is 17     The target sdk version is too old - writing the pm install command to STDOUT ... # Vi IMproved Touch 2.3 - target sdk version: 17 cat net.momodalo.app.vimtouch-5d7d195f2c9f5a0d04facb2dd060f73f.apk | adb shell pm install --bypass-low-target-sdk-block -S 3611858 Processing the file "org.jessies.dalvikexplorer-960d5c3572ca30db1e7af1e94ad64819.apk" ...     This is the app "Dalvik Explorer 3.9"; the target sdk version is 19     The target sdk version is too old - writing the pm install command to STDOUT ... # Dalvik Explorer 3.9 - target sdk version: 19 cat org.jessies.dalvikexplorer-960d5c3572ca30db1e7af1e94ad64819.apk | adb shell pm install --bypass-low-target-sdk-block -S 102780 Processing the file "ws.xsoh.etar-e5fac22dc5e52662d452440be682af76.apk.gz" ...     This is the app "Etar 1.0.42"; the target sdk version is 34     The target sdk version okay 3 apk file(s) processed 2 apk file(s) found with outdated target sdk The pm commands to install the apks are stored in the file "standard output (use redirection for stdout (> <filename>) to write to a file)" The list of apks including the target sdk size were written to "/tmp/apks.773891.lst" [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/data_backup/ASUS_I006D/TitaniumBackup ] $

The commands in blue are the commands to install the apks manually.

check_apks.sh writes all messages to STDERR and can therefore be used with a redirection for STDOUT to create a file that only contains the commands for manual installation of the apks:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/data_backup/ASUS_I006D/TitaniumBackup ] $ check_apks.sh   >/tmp/install_apks.sh check_apks.sh - check the target sdk version of apk files NOT running in the Android OS Creating "pm install" commands that must be executed on the PC Searching for apk files with a target sdk value less then 23 The pm commands to install the apks will be written to "/tmp/install_apks.sh" Processing the apk files in the directory "/data/backup/ASUS_ZENFONE8/data_backup/ASUS_I006D/TitaniumBackup" ... 3 apk(s) found Creating the list of latest apk files ..: ... 3 unique apk(s) found Processing the file "net.momodalo.app.vimtouch-5d7d195f2c9f5a0d04facb2dd060f73f.apk" ...     This is the app "Vi IMproved Touch 2.3"; the target sdk version is 17     The target sdk version is too old - writing the pm install command to STDOUT ... Processing the file "org.jessies.dalvikexplorer-960d5c3572ca30db1e7af1e94ad64819.apk" ...     This is the app "Dalvik Explorer 3.9"; the target sdk version is 19     The target sdk version is too old - writing the pm install command to STDOUT ... Processing the file "ws.xsoh.etar-e5fac22dc5e52662d452440be682af76.apk.gz" ...     This is the app "Etar 1.0.42"; the target sdk version is 34     The target sdk version okay 3 apk file(s) processed 2 apk file(s) found with outdated target sdk The pm commands to install the apks are stored in the file "/tmp/install_apks.sh" The list of apks including the target sdk size were written to "/tmp/apks.774194.lst" [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/data_backup/ASUS_I006D/TitaniumBackup ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/data_backup/ASUS_I006D/TitaniumBackup ] $ cat /tmp/install_apks.sh # Vi IMproved Touch 2.3 - target sdk version: 17 cat net.momodalo.app.vimtouch-5d7d195f2c9f5a0d04facb2dd060f73f.apk | adb shell pm install --bypass-low-target-sdk-block -S 3611858 # Dalvik Explorer 3.9 - target sdk version: 19 cat org.jessies.dalvikexplorer-960d5c3572ca30db1e7af1e94ad64819.apk | adb shell pm install --bypass-low-target-sdk-block -S 102780 [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/data_backup/ASUS_I006D/TitaniumBackup ] $
The script prints an entry for each apk with outdated configuration - in most cases it is not useful to restore all apks (e.g. there might be some apks in the backup that are already deleted from the phone) so I recommend checking the pm install commands in the file created by check_apks.sh and deleting or commenting out all unnecessary entries.
Since the name of an app can usually not be determined from the name of the apk file, check_apks.sh always prints the name and, if it can be determined from the apk file, the version of the app.

After all apks with outdated sdk target versions have been installed via the "pm install" command, you can restore the remaining apps from the backup using the menu points for restoring missing apps in Titanium backup.

Note that the commands for manual installation of the apk file are preceded by "adb shell " when the script is executed on a PC so that the apks can be installed via adb on the PC (use the script parameter "for_android" to disable that feature).

The default Android SDK version used for the check is hardcoded in the script:

# required target sdk version for the running Android OS # REQUIRED_MINIMUM_TARGET_SDK_VERSION=23

To check for another target SDK version use the script parameter skd=nnn, e.g. to create the "pm install" command for all apk files in a directory, execute the script with the parameter "sdk=100"

Notes:

check_apk.sh never installs or changes anything on the phone.

How to get an aapt executable for the Android OS

A git repository with scripts and config files to create the SDK tools for the aarch64 CPU is here : https://github.com/lzhiyong/android-sdk-tools

I have tested the instructions on this page, but unfortunately the exact version of the SDK and NDK used to create the instructions is missing, and with the current versions of the kits the configuration files used do not work out of the box.

A compiled SDK for running in the Android OS on a phone with aarch64 CPU is available here:

https://github.com/Lzhiyong/termux-ndk/releases/download/android-sdk/android-sdk-aarch64.zip

To extract only the aapt executable for the Android OS from the zip file do:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/img/android ] $ unzip -p android-sdk-aarch64.zip android-sdk/build-tools/34.0.0/aapt >./aapt [ OmniRom 14 Dev - xtrnaw7@t15g /data/img/android ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/img/android ] $ adb push aapt /data/local/tmp/ aapt: 1 file pushed, 0 skipped. 371.2 MB/s (3897896 bytes in 0.010s) [ OmniRom 14 Dev - xtrnaw7@t15g /data/img/android ] $

Test in an adb shell:

ASUS_I006D:/ $ file /data/local/tmp/aapt /data/local/tmp/aapt: ELF executable, 64-bit LSB arm64, static, for Android 30, built by NDK r24 (8215888), stripped ASUS_I006D:/ $ ASUS_I006D:/ $ chmod 755 /data/local/tmp/aapt ASUS_I006D:/ $ ASUS_I006D:/ $ /data/local/tmp/aapt v Android Asset Packaging Tool, v0.2-289 ASUS_I006D:/ $

see also https://hax4us.github.io/2021-11-22-install-android-sdk-in-termux/

Use this link to download the script check_apks.sh

Notes

see also How to install packages (apk files) for Android via script

The script check_apks.sh is a quick and dirty work and not optimized for performance. Therefor I recommend to execute it on a PC if applicable.

History of this entry

11.04.2024 /bs

initial release

----

Some hints about adb and fastboot usage in Linux

URL: https://xdaforums.com/t/some-hints-about-adb-and-fastboot-usage-in-linux.4673460/

Some hints about adb and fastboot usage in Linux

Prerequisites for accessing a phone connected via USB port

Note

If access to a phone connected via USB already works, you can skip this part of this post.

On most Linux distributions special udev rules are required to access phones connected via USB ports. These udev rules should be automatically added while installing the packages with the Android tools. To check this, search for a file called android something in the directory /etc/udev/rules.d , example:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $ ls -l /etc/udev/rules.d total 80 -rw-r--r--. 1 root root 1926 Jan 28 13:10 40-libsane.rules lrwxrwxrwx. 1 root root 45 May 15 11:39 51-android.rules -> /usr/share/doc/android-tools/51-android.rules-rw-r--r--. 1 root root 73728 May 15 11:18 51-android.rules:q -rw-r--r--. 1 root root 1926 Jan 28 13:10 S99-2000S1.rules [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $

If the udev config to access phones via USB is missing, add the udev rules manually, e.g. on Fedora 39 after installing the rpm package with the Android tools (android-tools) this can be done using these commands:

sudo ln -s /usr/share/doc/android-tools/51-android.rules /etc/udev/rules.d

and request the udev daemon to re-load the udev rules:

sudo udevadm control --reload sudo udevadm trigger

Since the devices for USB devices are created and destroyed dynamically, you must physically disconnect and reconnect the phone to activate the new udev rule.

To check, that your phone is supported by the udev config use these instructions:

Attach your phone via USB and list the attached USB devices to get the Vendor ID for the phone, example (for an ASUS Zenfone 8):
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $ lsusb | grep Zenfone
Bus 003 Device 097: ID 0b05:7770 ASUSTek Computer, Inc. Zenfone 8
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $

The Vendor ID necessary for the udev rule is the first hexadecimal string after ID, in this example 0b05.

Now check, that the Vendor ID for your phone is configured in the udev config file:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $ grep -i 0b05 /etc/udev/rules.d/51-android.rules SUBSYSTEM=="usb", ATTR{idVendor}=="0b05", MODE="0666", GROUP="plugdev" [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $

If the config is missing, add it manually and reload the udev config (see above).

Next check, that your userid is member of the group configured in the udev config file (in this example the group is called plugdev):

xtrnaw7@t15g:~$ id uid=1000(xtrnaw7) gid=1000(xtrnaw7) groups=1000(xtrnaw7),6(disk),10(wheel),967(adbusers),970(vboxusers),1002(tftp),1003(plugdev) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 xtrnaw7@t15g:~$

Add your userid to that group if it's not yet member of the group:

sudo gpasswd -a $(whoami) plugdev

Newly configured groups for a user are only active for new sessions; I therefore recommend restarting the PC after changing the group configuration to ensure that all processes running for this user ID use the new group.

Note

A repository with udev rules for Android devices that claims to support most of the Android phones is available here: https://github.com/M0Rf30/android-udev-rules

The adb daemon

A running adb daemon under Linux is required to access the phone via USB using adb.

If the adb daemon is not running when adb is executed, it is started automatically; example: [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/scripts_on_linux ] $ ps -ef | grep -v grep | grep " adb " [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/scripts_on_linux ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/scripts_on_linux ] $ adb shell uname -a * daemon not running; starting now at tcp:5037 * daemon started successfullyLinux localhost 5.4.268-qgki-perf-gb2eb9881deaa-dirty #1 SMP PREEMPT Mon Feb 19 18:27:19 EST 2024 aarch64 Toybox [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/scripts_on_linux ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/scripts_on_linux ] $ ps -ef | grep -v grep | grep " adb " xtrnaw7   983862       1 0 09:54 ?        00:00:00 adb -L tcp:5037 fork-server server --reply-fd 4 [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/scripts_on_linux ] $
To start the adb manually, use the adb parameter start-server; for example, to start the adb daemon explicitly as the root user:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ sudo adb start-server * daemon not running; starting now at tcp:5037 * daemon started successfully [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] # check the result [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ps -ef | grep -i " adb " root     1003277       1 0 11:44 ?        00:00:00 adb -L tcp:5037 fork-server server --reply-fd 4 xtrnaw7 1003322   12411 0 11:44 pts/3    00:00:00 grep --color=auto -i adb [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

The running adb daemon serves all adb requests from all users on the PC, regardless of which user started it, for example:

ums@t15g:~$ ps -ef | grep -v grep | grep " adb " xtrnaw7   992597 992596 0 10:41 ?        00:00:00 adb -L tcp:5037 fork-server server --reply-fd 4 ums@t15g:~$ ums@t15g:~$ id uid=1001(ums) gid=1001(ums) groups=1001(ums) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 ums@t15g:~$ ums@t15g:~$ adb shell uname -a Linux localhost 5.4.268-qgki-perf-gb2eb9881deaa-dirty #1 SMP PREEMPT Mon Feb 19 18:27:19 EST 2024 aarch64 Toybox ums@t15g:~$

Every user on the PC can kill the adb daemon using the approbiate parameter for the adb command, for example:
[ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ ps -ef | grep -v grep | grep -i " adb " root      987419       1 0 10:12 ?        00:00:00 adb -L tcp:5037 fork-server server --reply-fd 4 [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ id uid=1000(xtrnaw7) gid=1000(xtrnaw7) groups=1000(xtrnaw7),6(disk),10(wheel),970(vboxusers),1002(tftp) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ adb kill-server [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ ps -ef | grep -v grep | grep -i " adb " [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $

The connection via adb between the PC and the phone is secured by an SSL key. The on SSL key used for this connection is the file adbkey in the directory .android in the home directory of the user who started the adb daemon on the PC, example:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/scripts_on_linux ] $ ps -ef | grep -v grep | grep " adb " xtrnaw7   983862       1 0 09:54 ?        00:00:00 adb -L tcp:5037 fork-server server --reply-fd 4 [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/scripts_on_linux ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/scripts_on_linux ] $ grep xtrnaw7 /etc/passwd xtrnaw7:x:1000:1000:Bernd Schemmer:/home/xtrnaw7:/bin/bash [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/scripts_on_linux ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/scripts_on_linux ] $ ls -al /home/xtrnaw7/.android/ total 128 drwxr-x---. 4 xtrnaw7 xtrnaw7 4096 Apr 29 17:07 . drwx------. 66 xtrnaw7 xtrnaw7 12288 May 3 08:52 .. -rw-------. 1 xtrnaw7 xtrnaw7 1704 Mar 31 2022 adbkey -rw-r--r--. 1 xtrnaw7 xtrnaw7   722 Apr 2 11:07 adbkey.pub [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/Android/scripts_on_linux ] $

The SSL key in the directory ~/.android is created automatically by the adb the first time it is started for this user:

ums@t15g:~$ id uid=1001(ums) gid=1001(ums) groups=1001(ums) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 ums@t15g:~$ ums@t15g:~$ ls -l $HOME/.android total 0 ums@t15g:~$ ums@t15g:~$ adb shell uname -a * daemon not running; starting now at tcp:5037 * daemon started successfully ums@t15g:~$ ums@t15g:~$ ums@t15g:~$ ls -l $HOME/.android total 12 -rw-r--r--. 1 ums ums   12 May 3 10:05 adb.5037 -rw-------. 1 ums ums 1704 May 3 10:05 adbkey -rw-r--r--. 1 ums ums 717 May 3 10:05 adbkey.pub ums@t15g:~$

The executable openssl can be used to check or view the key (well, the infos printed by openssl for the key are not really useful ...):

openssl rsa -in adbkey -text -noout

[ OmniRom 14 Dev - xtrnaw7@t15g ~/.android ] $ openssl rsa -in adbkey -text -noout Private-Key: (2048 bit, 2 primes) modulus: 00:b6:eb:29:14:2c:f7:8f:ee:48:75:a1:4f:35:ba: 4a:30:f2:0c:8f:f1:70:20:37:3e:0c:14:04:5b:da: 92:df:16:d3:8b:a7:d2:96:d9:11:81:d8:10:93:42: 7d:39:a6:a4:bb:48:ad:c7:c8:f9:7b:75:07:8d:53: 1c:ef:04:15:9b:0a:c1:d6:c7:53:a6:c6:fe:46:da: bc:69:45:01:68:07:0d:72:9d:67:28:4e:48:c6:17: f4:1d:f0:14:38:6e:e1:c8:d0:60:6f:30:99:a9:ba: 9a:f7:5a:8f:18:ce:b0:cc:92:b0:21:81:9f:a0:f7: 06:8e:52:58:07:5b:78:f8:9f:3f:e2:d0:0f:7e:69: 45:c3:34:e8:51:71:2b:af:e0:9e:af:2b:58:5d:ff: 00:12:4c:f5:10:f7:01:cd:89:38:6b:15:e4:60:2f: b4:20:43:b9:53:96:5d:b8:8a:43:0a:07:20:1f:76: d3:ce:d2:bc:b4:76:7a:b9:c3:71:b4:6b:39:bd:f7: 2b:26:d8:94:18:b0:91:c6:13:43:de:7d:d1:5e:7b: 9c:be:b0:a9:f9:5c:f8:0e:74:c0:7d:15:62:7a:45: 3b:2e:12:76:62:37:47:31:47:20:77:5c:f9:2d:e4: 0d:6f:2f:c8:07:f9:73:2e:0d:f3:e3:ae:ce:76:33: dc:55 publicExponent: 65537 (0x10001) privateExponent: 20:f1:0f:c8:50:e2:d2:9a:64:95:3e:94:2f:c6:59: 57:20:38:2f:f2:18:b1:cc:91:11:86:c7:54:2d:74: 56:d5:db:0a:23:12:93:55:0b:48:99:7b:3e:b1:f2: 30:60:38:f9:7b:78:c4:6a:86:b7:7b:97:7d:15:93: 37:de:41:ef:d6:8b:9b:1c:f1:8d:2e:f8:1b:15:88: 69:e2:e2:02:74:86:b0:f5:f4:de:76:de:b7:42:18: 16:0e:26:ee:14:d5:f7:9e:c3:47:32:f6:f1:70:a8: 38:d4:a7:c1:9d:73:8c:9b:fc:39:44:89:55:69:37: 56:89:30:ce:3e:64:76:4d:95:b6:ee:48:54:00:85: 39:c7:98:63:fa:b1:7f:62:37:82:ea:f0:04:00:97: e4:82:04:e2:49:d3:4e:d4:1c:16:1f:e2:2b:ea:ba: 1b:63:24:9c:ed:6a:44:9c:dc:b2:c4:49:cd:c1:1e: 7f:45:5f:ab:2e:f3:d4:b5:50:e4:c7:59:f7:e1:d7: af:79:3c:28:e6:33:28:66:34:3c:7f:03:b8:db:89: 41:c2:30:b1:02:29:10:11:d0:1b:96:5a:73:dc:c6: 98:d4:bf:ee:e6:99:7b:fd:21:19:8c:2e:c6:6c:df: dc:0b:fc:1c:64:8f:77:a7:71:1f:0e:6d:a2:a5:c3: cf prime1: 00:da:b7:52:ee:2e:f0:bf:0a:a9:9e:94:1f:1a:e7: 1a:e6:ee:06:fd:ef:a4:9e:70:a4:3e:c2:e3:fb:49: 81:90:ca:f5:39:e7:a0:b4:90:3b:5f:4c:46:c2:15: 7b:07:8f:9b:47:90:bf:11:8e:5b:66:18:3d:7d:e6: 9b:b1:76:fc:2b:12:68:bd:1f:7a:64:28:42:69:41: df:f1:e0:a3:8f:a7:59:63:0b:a8:fb:20:0e:b8:65: a1:51:bb:ff:03:33:bc:e3:dc:a2:9e:72:70:27:55: 23:2d:98:a1:30:07:8d:66:84:e2:fd:41:1e:ff:8e: 19:12:51:63:4c:ad:1f:a0:ab prime2: 00:d6:19:a6:08:4e:60:f4:6c:8b:8b:6d:53:29:53: 3f:e1:44:4d:f2:25:33:38:c6:8b:9b:27:22:ba:26: d1:4a:33:40:f2:0d:a6:32:10:e9:3c:a5:c2:15:d1: 63:1f:ad:e4:87:82:bc:4e:ed:41:09:b4:ba:e3:ff: 2c:2f:34:a2:11:4c:ea:2d:aa:af:d9:32:59:a8:e0: a8:90:7a:21:f5:cd:51:39:cf:ee:fa:93:87:f7:0e: 96:b7:b3:71:ed:17:ce:82:99:4c:dc:b4:8a:75:88: e6:bf:5f:96:f0:0f:52:62:39:44:57:1d:a4:b7:88: 56:78:83:b2:3f:f6:ec:76:ff exponent1: 62:69:d4:ee:09:bf:18:27:43:15:70:ce:e2:3b:15: 16:ce:09:e5:5f:5c:72:52:7b:05:26:8a:90:fa:8d: 4b:4a:97:b7:5b:6b:2b:74:2e:d2:fd:ae:65:0c:67: 54:8a:a0:b3:3e:05:4b:70:03:de:57:8f:eb:c3:c5: 50:c0:1a:4d:83:36:0f:cb:47:36:24:0b:65:f3:57: 42:2d:f3:4a:e3:61:4c:38:e5:eb:41:8c:7e:1a:74: d1:23:47:66:f8:73:c9:0f:f1:38:5c:83:ec:9e:d9: 7a:5d:d0:41:9d:54:59:c9:f7:9d:3f:1d:24:9e:9a: d7:6c:39:c3:39:af:1e:8b exponent2: 77:94:83:c3:87:e3:67:21:69:1f:4a:c4:74:04:67: 5d:6e:45:0a:c5:f4:41:f5:c2:eb:b9:84:0f:ec:b6: 60:77:18:03:19:d4:85:8a:24:7b:17:29:bb:fd:e3: 1b:42:88:ad:97:3b:8d:e9:bd:b5:aa:17:e8:58:11: 59:9c:50:18:d1:98:ca:40:d7:4e:bc:6c:8f:82:4b: 23:c6:d3:48:f2:90:37:76:07:c8:34:b6:70:2a:d9: cb:68:92:6d:16:2a:e5:8e:b1:fb:63:6b:22:12:80: ae:ca:b9:07:03:c7:c3:cc:b0:7a:e7:b7:a4:3a:a5: f1:51:bd:31:34:bd:06:09 coefficient: 00:cf:e1:3b:22:2c:3b:95:77:d1:e0:08:a4:75:36: cc:de:4b:60:7f:ee:82:a0:e1:eb:93:89:90:e4:c6: 0c:cc:cd:57:78:cd:79:e5:67:b9:c3:85:17:d0:25: 00:d9:bf:c1:c4:f7:34:b8:6a:fa:61:c5:10:37:f9: c3:44:d5:48:3d:a5:a4:e3:a5:86:b3:0b:30:f2:f1: e8:21:4d:1f:6c:ca:9c:79:45:f8:86:11:68:ea:81: 32:66:d6:df:1b:49:d5:26:70:b4:59:ff:7c:62:78: 49:64:30:2f:11:19:e6:06:31:d8:cc:76:6f:84:7c: 04:03:39:1d:97:ba:74:fb:ef [ OmniRom 14 Dev - xtrnaw7@t15g ~/.android ] $

On the first connection from a new PC via adb, the phone prints the fingerprint of the SSL key used for the connection and prompts the user for confirmation, for example:

To check the fingerprint of the SSL key on the PC use this command

adb pubkey ${HOME}/.android/adbkey | awk '{ print $1}' | openssl base64 -A -a -d | openssl md5 -c | tr a-z A-Z

Example:

[ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ adb pubkey ${HOME}/.android/adbkey | awk '{ print $1}' | openssl base64 -A -a -d | openssl md5 -c | tr a-z A-Z MD5(STDIN)= F9:48:9A:7F:8E:E9:E6:C0:DB:83:C3:53:6D:4A:5C:97 [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $

(This command is "stolen" from this page: https://joachimschuster.de/posts/debug-on-device-rsa-fingerprint/ )

Notes:

The SSL key is neither protected by a private key nor user-dependent. Therefore, you can just copy the file adbkey to the directory .android in the home directdory of another user on the same or a different PC to enable adb connections for that user. You should therefore take care of the SSL keys.

~~For using the sideload feature of the binary~~ ~~twrp~~ ~~from~~ ~~TWRP~~ ~~the~~ ~~adb~~ ~~on the PC must be started by the user~~ ~~root~~ ~~(see~~ ~~How to install an OS image via sideload using the TWRP binary twrp~~)
With correct udev rules adb sideload also works if the adb was started by a non-root user.

In Fedora 39 (and probably in other Linux distributions also) the package with the Android tool creates a systemd service to start the adb daemon:

[ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ systemctl status adb â—‹ adb.service - Android Debug Bridge (adb) service Loaded: loaded (/usr/lib/systemd/system/adb.service; disabled; preset: disabled) Drop-In: /usr/lib/systemd/system/service.d â””â”€10-timeout-abort.conf Active: inactive (dead) since Fri 2024-05-03 11:02:48 CEST; 6s ago Duration: 6min 57.269s Process: 995228 ExecStart=/usr/bin/adb start-server (code=exited, status=0/SUCCESS) Process: 996171 ExecStop=/usr/bin/adb kill-server (code=exited, status=0/SUCCESS) Main PID: 995229 (code=exited, status=0/SUCCESS) CPU: 113ms May 03 10:55:48 t15g.isbs.de systemd[1]: Starting adb.service - Android Debug Bridge (adb) service... May 03 10:55:48 t15g.isbs.de adb[995228]: * daemon not running; starting now at tcp:5037 May 03 10:55:51 t15g.isbs.de adb[995228]: * daemon started successfully May 03 10:55:51 t15g.isbs.de systemd[1]: Started adb.service - Android Debug Bridge (adb) service. May 03 11:02:48 t15g.isbs.de systemd[1]: Stopping adb.service - Android Debug Bridge (adb) service... May 03 11:02:48 t15g.isbs.de systemd[1]: adb.service: Deactivated successfully. May 03 11:02:48 t15g.isbs.de systemd[1]: Stopped adb.service - Android Debug Bridge (adb) service. [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $
The service is not enabled by default:

[ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ systemctl is-enabled adb disabled [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $

The service starts the adb daemon as user root.

To start the adb daemon using the systemd service execute as user root:

systemctl start adb

To automatically start the adb daemon using the systemd service after every reboot execute as user root:

systemctl enable abd

In case you're using the systemd service for the adb daemon be aware that the SSL key used for the connection might not be the key from the directory ${HOME}/.android from the user root:

[ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ cat /usr/lib/systemd/system/adb.service # Systemd unit file for adb [Unit] Description=Android Debug Bridge (adb) service [Service] Type=forking ExecStart=/usr/bin/adb start-server ExecStop=/usr/bin/adb kill-server PrivateTmp=yes Environment=HOME=/var/lib/adb [Install] WantedBy=multi-user.target [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $

In this configuration the SSL key used for the adb connection is in the directory /var/lib/adb/.android:

[ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $ sudo ls -al /var/lib/adb/.android/ total 20 drwxr-x---. 2 root root 4096 Apr 29 23:37 . drwxr-xr-x. 3 root root 4096 Apr 29 23:37 .. -rw-r--r--. 1 root root 12 May 3 10:55 adb.5037 -rw-------. 1 root root 1704 Apr 29 23:37 adbkey -rw-r--r--. 1 root root 714 Apr 29 23:37 adbkey.pub [ OmniRom 14 Dev - xtrnaw7@t15g ~ ] $

On the phone the config files for the adb connections are stored in the directory /data/misc/adb :

ASUS_I006D:/ # id uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 ASUS_I006D:/ # ASUS_I006D:/ # ls -l /data/misc/adb/ total 8 -rw-r----- 1 system shell 722 2024-05-03 11:15 adb_keys -rw------- 1 system shell 810 2024-05-03 11:15 adb_temp_keys.xml ASUS_I006D:/ #

To restart the adb on the phone this command can be used on the PC:

adb shell su - -c setprop ctl.restart adbd

The command to reset the USB port on the phone is

svc usb resetUsbPort

e.g.

ASUS_I006D:/ $ svc usb resetUsbPort Use the default USB port: port0 Reset the USB port: port0 ASUS_I006D:/ $

Note that resetting the USB port does not end existing adb connections to the phone

Access to the phone connected via USB using fastboot

The udev configuration is also required to access a phone connected via USB with fastboot.

In many udev configurations for accessing a phone, the udev connections allow access via adb for non-root users, but not access via fastboot.

In this case, access to the phone via fastboot is only allowed for the root user, so any fastboot command executed by a non-root user must be preceded by "sudo", example:
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $ fastboot devices no permissions (user xtrnaw7 is not in the plugdev group); see [http://developer.android.com/tools/device.html] fastboot [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $

but
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $ sudo fastboot devices M6AIB760D0939LX fastboot [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $

If this is the case: Check the udev configuration for your phone; it should be as listed above:

SUBSYSTEM=="usb", ATTR{idVendor}=="0b05", MODE="0666", GROUP="plugdev"

and your userid must be member of the group configured for the USB device (plugdev in this example).

Update 04.07.2023

If access via fastboot does not work at all, try a different USB cable: Not all USB cables can be used for access via fastboot. If this does not work, use a USB 2.0 port for access - some USB 3.x ports cannot be used for access via fastboot.
If the PC does not have USB 2.x ports, try an external USB 2.0 hub.

see also:

adb documentation in the Android SDK

adb HowTo on XDA

adb man page

How to enable access via adb on a new installed OS

How to reset the USB port used to connect an Android_phone

How to enable adb via WiFi

Some hints about EDL mode

History of this entry

26.04.2024

initial release

23.05.2024

added infos about how to reset the USB port on the phone

04.07.2024

added trouble shooting infos to fix not working access via fastboot

How to add additional prebuild apks to a custom ROM

URL: https://xdaforums.com/t/how-to-add-additional-prebuild-apks-to-a-custom-rom.4671911/

How to add additional prebuild apks to a custom ROM

Sometimes it makes sense to install additional predefined applications as part of the operating system.

For ROMs for which you compile the installation image yourself, adding prebuild applications to a CustomROM can be done as follows.

Notes

There are already some websites with instructions on this topic (e.g. here: https://adityatelange.in/blog/aosp/aosp-adding-prebuilt-apk/), but I didn't find all the information needed to add the apks and all the errors encountered during testing in one place, so I created this post.

All relative paths in the following instructions refer to the directory with the repositories for the ROM

The following example is for adding the app ChangeLog as prebuild app to the OmniROM.

This is certainly not very useful, but is only intended as an example (and it has been proven to work ...)

Note that the directory for the prebuild apps, vendor/omni/prebuild/app in the OmniROM, may be different in other Custom ROMS -- see How to add the ASUS Hardware Testtool as prebuild app for the StatiXOS ROM for an example.

1. Create a directory for the new app in the directory vendor/omni/prebuilt/app in the directory tree with the source files for the ROM:

# change the working directory to the directory with the repositories for the OmniROM # cd /data/develop/android/OmniROM_14.0/ # create the new directory # mkdir vendor/omni/prebuilt/app/changelog

2. Copy the apk file to the new created directory (the source directory on your PC will be different I guess ...):

cp /data/backup/Android/EssentialApps/com.bytehamster.changelog_v49.apk vendor/omni/prebuilt/app/changelog/com.bytehamster.changelog_v49.apk

3. Create the necessary Android.mk file for the new application. Either create the file manually or copy the Android.mk file from the directory of another app and edit the file contents:

cp vendor/omni/prebuilt/app/OmniStore/Android.mk vendor/omni/prebuilt/app/changelog/Android.mk

When done the file should look like this:
LOCAL_PATH:= $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := ChangeLog LOCAL_SRC_FILES := com.bytehamster.changelog_v49.apk LOCAL_MODULE_CLASS := APPS LOCAL_MODULE_TAGS := optional LOCAL_CERTIFICATE := PRESIGNED LOCAL_DEX_PREOPT := false LOCAL_SYSTEM_EXT_MODULE := true include $(BUILD_PREBUILT)

Be careful not to use leading or trailing whitespaces in the file - the make tools for Android don't like them and produce strange error message for these whitespaces.

And for the records because I only found this infos by accident (see also https://android.googlesource.com/platform/build/+/acd93c7f6df7cd70d12622bf9fdb4af5d8f0eb24/core/prebuilt.mk)

Known values for LOCAL_CERTIFIACTE are (as far as I know ...):

Value	Description
PRESIGNED	the apk file is already signed
platform	the build scripts will sign the apk with the platform key this will be converted to: -> LOCAL_CERTIFICATE := build/make/target/product/security/platform
shared	"a key for things that are shared in the home/contacts process." this will be converted to: -> LOCAL_CERTIFICATE := build/make/target/product/security/shared
media	the build scripts will sign the apk with the key used for packages in media/download this will be converted to: -> LOCAL_CERTIFICATE := build/make/target/product/security/media
vendor/example/certs/app	the bulild scripts will sign the apk with the key found in the local directory vendor/example/certs this will be converted to: -> LOCAL_CERTIFICATE := vendor/example/certs/app
testkey	a key used for all other packages default this will be converted to: -> LOCAL_CERTIFICATE := build/make/target/product/security/testkey
EXTERNAL	"The magic string "EXTERNAL" means this package will be signed with the test key throughout the build process, but we expect the final package to be signed with a different key." -> LOCAL_CERTIFICATE := build/make/target/product/security/testkey

After converting the value like described in the table above the make system creates the final file names of the certificates to use :

PACKAGES.$(LOCAL_MODULE).PRIVATE_KEY := $(LOCAL_CERTIFICATE).pk8
PACKAGES.$(LOCAL_MODULE).CERTIFICATE := $(LOCAL_CERTIFICATE).x509.pem

4. Finally add the new app to the file vendor/omni/config/packages.mk, e.g.:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ grep -B2 -A1 ChangeLog vendor/omni/config/packages.mk PRODUCT_PACKAGES += \ ChangeLog [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

Note:

If you use a name in PRODUCT_PACKAGES that doesn't exist the build system silently ignores your mistake.

Now recreate the OS image and you're done.

Notes:

To add SELinux policies for the new app; create a new .te file in one of the directories named sepolicy , e.g.:

echo "allow system_app proc_stat:file { ioctl read getattr lock map open watch watch_reads };" >device/asus/zenfone8/sepolicy/vendor/asus_hardware_testtool.te

see How to add missing SELinux policies dynamically using Magisk for some details regarding SELinux policies.

Trouble Shooting

Error message like this

vendor/omni/prebuilt/app/MyTerminal/Android.mk: error: MyTerminal: Invalid characters in module stem $LOCAL_INSTALLED_MODULE_STEM$:
while creating the OS image are most probably caused by leading or trailing whitespaces in the file Android.mk.

If you get an error message like this:

FAILED: out/target/product/zenfone8/obj/APPS/MyTerminal_intermediates/enforce_uses_libraries.status /bin/bash -c "(rm -f out/target/product/zenfone8/obj/APPS/MyTerminal_intermediates/enforce_uses_libraries.status ) && (build/soong/scripts/manifest_check.py --enforce-uses-libraries --enforce-uses-libraries- status out/target/product/zenfone8/obj/APPS/MyTerminal_intermediates/enforce_uses_libraries.status --aapt out/host/linux-x86/bin/aapt2 vendor/omni/prebuilt/app/MyTerminal/com.t ermoneplus_501.apk )" error: mismatch in the <uses-library> tags between the build system and the manifest: - required libraries in build system: [] vs. in the manifest: [] - optional libraries in build system: [] vs. in the manifest: [androidx.window.extensions, androidx.window.sidecar] - tags in the manifest (vendor/omni/prebuilt/app/MyTerminal/com.termoneplus_501.apk): uses-library-not-required:'androidx.window.extensions' uses-library-not-required:'androidx.window.sidecar'note: the following options are available: - to temporarily disable the check on command line, rebuild with RELAX_USES_LIBRARY_CHECK=true (this will set compiler filter "verify" and disable AOT-compilation in dexpreopt) - to temporarily disable the check for the whole product, set PRODUCT_BROKEN_VERIFY_USES_LIBRARIES := true in the product makefiles - to fix the check, make build system properties coherent with the manifest - for details, see build/make/Changes.md and https://source.android.com/devices/tech/dalvik/art-class-loader-context 11:30:28 ninja failed with: exit status 1 There were 3 actions that completed after the action that failed. See verbose.log.gz for their output.

while building the OS image use the instructions below to fix the error.

The reason for the error message are instructions for required libraries in the manifest file of the apk file, which are missing in the Android.mk file for adding the app as a prebuild app.

To fix the issue:

First double check the list of required libraries in the Manifest from the apk file using aapt:
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ APK=vendor/omni/prebuilt/app/MyTerminal/com.termoneplus_501.apk [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ ../android_sdk/build-tools/34.0.0/aapt dump badging $APK | grep uses-library uses-library-not-required:'androidx.window.extensions' uses-library-not-required:'androidx.window.sidecar'[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

The list should be equal to the list of tags in the error message.

Now to fix this issue add another line to the Android.mk file with these requirements for the apk fie:

LOCAL_OPTIONAL_USES_LIBRARIES :=androidx.window.extensions androidx.window.sidecar

Note that the order of the values for LOCAL_OPTIONAL_USES_LIBRARIES must match the order from the manifest in the apk file.

If the new app does not appear to be installed on the phone after installing the new operating system image, first check whether the apk file is present on the phone, e.g.:

ASUS_I006D:/ $ ls -l /system_ext/app/ChangeLog/ total 1096 -rw-r--r-- 1 root root 1121030 2009-01-01 01:00 ChangeLog.apk ASUS_I006D:/ $
The new apk should be in the directory /system_ext/app/<appname> because of this entry in the Android.mk file: LOCAL_SYSTEM_EXT_MODULE := true; without this statement the apk file should be in the directory /system/app/<appname> or, if the statement LOCAL_MODULE_PATH := $(TARGET_OUT_DATA) is used in the Android.mk file, in the directory /data/app/<appname>.

If the file does not exist something went wrong creating the OS image - check the log file of the build process, e.g:

gzip -cd out/verbose.log.gz | grep -i ChangeLog.apk

If the file exists check the logcat output on the phone for error messages regarding the apk file:

logcat -d | grepChangeLog.apk

To prove that the file can be installed, try to install the apk manually in an adb shell:

CUR_SIZE=$( ls -l /system_ext/app/ChangeLog/ChangeLog.apk | awk '{ print $5}' ) cat/system_ext/app/ChangeLog/ChangeLog.apk | pm install -S ${CUR_SIZE}

Error message like this in the logcat on the phone regarding the new application:

05-12 09:02:43.968 1265 1265 W PackageManager: Failed to scan /system_ext/app/MyTerminal: No APK Signature Scheme v2 signature in package /system_ext/app/MyTerminal/MyTerminal.apk

are most likely there because the Android make tools rebuilt the apk file but did not re-sign the file and therefor the apk file is not signed anymore.
To fix it, change the value for the variable LOCAL_CERTIFICATE in the file Android.mk to force the Android tools to re-sign the apk file.

Or add the entry

LOCAL_REPLACE_PREBUILT_APK_INSTALLED := $(LOCAL_PATH)/com.bytehamster.changelog_v49.apk

to the file Android.mk .

Notes

In case you do not have the source code of the ROM you might use the method described in the post How to "simulate" prebuild apps using Magisk to simulate a prebuild app.

see How to add the ASUS Hardware Testtool as prebuild app for a Custom ROM for an example for adding a prebuild app
see How to add the ASUS Hardware Testtool as prebuild app for the StatiXOS ROM for an example for adding a prebuild app

History of this entry

11.05.2024

initial release

26.05.2024

added links to other posts related to this topic

How to "simulate" prebuild apps using Magisk

URL: https://xdaforums.com/t/how-to-simulate-prebuild-apps-using-magisk.4671914/

How to "simulate" prebuild apps using Magisk

For the development of my scripts for the automatic installation and configuration of the Android operating system it would have been helpful to have a terminal emulator available immediately after the first reboot following the installation of the operating system.

Unfortunately most terminal apps for Android that are available on F-Droid, for example, cannot be installed as prebuild app in a CustomROM using the "official" method described in the post How to add additional prebuild apks to a custom ROM without additional configuration adjustments, most probably because they are too old (the installations works somehow but the apps crashes shortly after starting).

And of course this method does not work for ROMs that you do not compile yourself.

For ROMs for which you do not have the source code or which you do not want to compile yourself, you can therefore use this approach:

Use Magisk to create a new init .rc file like described in the entry How to enable access via adb on a new installed OS. Add this code to the script executed by the new init .rc file:

# --------------------------------------------------------------------- # install additional apk files found in the directory /data/recovery # APK_INSTALL_STATUS_FILE="/data/recovery/additional_apks_already_installed" if [ ! -r "${APK_INSTALL_STATUS_FILE}" ] ; then echo "Additional apk files installed at $( date) " >"${APK_INSTALL_STATUS_FILE}" for CUR_APK_FILE in /data/recovery/*.apk; do [[ ${CUR_APK_FILE} == *\** ]] && break
echo "Installing the apk file \"${CUR_APK_FILE}\" ..." CUR_APK_FILE_SIZE=$( ls -l "${CUR_APK_FILE}" | awk "{ print \$5 }" ) cat ${CUR_APK_FILE} | pm install -S "${CUR_APK_FILE_SIZE}" if [ $? -eq 0 ] ; then echo "\"${CUR_APK_FILE}\" successfully installed" | tee -a "${APK_INSTALL_STATUS_FILE}" else echo "ERROR: \"${CUR_APK_FILE}\" not installed" | tee -a "${APK_INSTALL_STATUS_FILE}" fi done else echo "Additional apk files already installed" cat "${APK_INSTALL_STATUS_FILE}" fi

Then copy the apk files that should be installed to the directory /data/recovery after the installation of the new OS but before the first reboot.

Note:

see the script enable_adb_using_magisk.sh for a working example using this approach.

See How to fix a hanging restore in Titanium Backup for installing "outdated" apks in Android 14 or newer.

History of this entry

15.05.2024

initial release

19.05.2024

added code to avoid errors if no apks to install exist

How to re-sign an apk file

URL: https://xdaforums.com/t/how-to-re-sign-an-apk-file.4671724/

How to re-sign an apk file

Some permissions in Android are only granted to an application if the apk file is signed with the certificate used to sign the OS image file
(see Instructions to use the ASUS Hardware Testtool for an example).

Therefore, to use this type of application on a self compiled OS image, the apk file must be signed with the certificate used for the OS image.

Here are the instructions how to do this:

In this example we re-sign the file MyTerminal.apk with the certificate used to sign a self-created OS image:

First remove the existing certificate in the package :

zip -d MyTerminal.apk META-INF/16A6B69C.SF META-INF/16A6B69C.RSA

Notes:

The name of the files with the certificate might be different in other packages; use a command like this to list the filenames for the certificate in the apk file:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0/vendor/omni/prebuilt/app/MyTerminal ] $ unzip -t MyTerminal.apk | egrep ".SF|.RSA"     testing: META-INF/16A6B69C.SF     OK     testing: META-INF/16A6B69C.RSA    OK [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0/vendor/omni/prebuilt/app/MyTerminal ] $

Now re-align the ZIP file:

ZIPALIGN="/data/develop/android/otatools/bin/zipalign" ${ZIPALIGN} 4 MyTerminal.apk MyTerminal.apk.aligned mv MyTerminal.apk.aligned MyTerminal.apk # Test the result (there should be NO output from this command): ${ZIPALIGN}-c 4 MyTerminal.apk

Next re-sign the apk file :
APKSIGNER="/data/develop/android/android_sdk/build-tools/34.0.0/lib/apksigner.jar" DIR_WITH_KEYS="/data/develop/android/OmniROM_14.0/build/make/target/product/security/" java -jar ${APKSIGNER} sign --key ${DIR_WITH_KEYS}/platform.pk8 --cert ${DIR_WITH_KEYS}/platform.x509.pem   MyTerminal.apk

To check the certificate use the commands

# print the certificate used to sign the apk file # openssl x509 -in /data/develop/android/OmniROM_14.0/build/make/target/product/security//platform.x509.pem -text -noout # print the certificate now used in the apk file # unzip -p MyTerminal.apk META-INF/PLATFORM.RSA | keytool -printcert

and compare the result, e.g.:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0/vendor/omni/prebuilt/app/MyTerminal ] $ openssl x509 -in /data/develop/android/OmniROM_14.0/build/make/target/product/security//platform.x509.pem -text -noout | grep -A1 " Serial Number:" | tail -1 | tr -d ":"             7947c8af38436b2da799707938319b7074d7b7e4 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0/vendor/omni/prebuilt/app/MyTerminal ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0/vendor/omni/prebuilt/app/MyTerminal ] $ keytool -printcert -jarfile MyTerminal.apk | grep -i serial | cut -f2 -d ":" 7947c8af38436b2da799707938319b7074d7b7e4 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0/vendor/omni/prebuilt/app/MyTerminal ] $

Note:

The file apksigner.jar and the executable zipalign are part of the Android SDK command line tools
keytool is part of the standard Java installation; openssl is a standard Linux executable.

A simple script to sign an apk file can be downloaded from my webpage : sign_apk.sh

Before using the script you must correct the values for some variables in the script to match your environment:

APKSIGNER="/data/develop/android/Sdk/build-tools/33.0.0/lib/apksigner.jar"ZIPALIGN="/data/develop/android/otatools/bin/zipalign"ZIP="/usr/bin/zip"# certificate used to sign the apk file#KEY_DIRECTORY="/data/develop/android/security"PK8_FILE="${KEY_DIRECTORY}/platform.pk8"PEM_FILE="${KEY_DIRECTORY}/platform.x509.pem"

How to add missing SELinux policies dynamically using Magisk

URL: https://xdaforums.com/t/how-to-add-missing-selinux-policies-dynamically-using-magisk.4671716/

How to add missing SELinux policies dynamically using Magisk

If an application in Android does not work due to missing SELinux rules this can be fixed on phones with installed Magisk using the binary magiskpolicy from Magisk.

To check whether an application fails due to missing SELinux rules use this approach:

Open an adb shell, become root user, and temporarily deactivate SELinux with the command:

setenforce 0

Now start the application: If the application now works, the SELinux rules for this application is missing.

In this case to find the missing SELinux rules do:

# enable SELinux again setenforce 1
run logcat via adb shell on the Linux PC:

adb shell "logcat -c ; logcat | grep denied "

Now start the application on the phone.

There should be a "denied" message like this about the missing SELinux rule in the output of the running logcat command, example:

05-13 21:46:24.935 5257 5257 W us.atd.smmitest: type=1400 audit(0.0:121): avc: denied { read } for name="stat" dev="proc" ino=4026532098 scontext=u:r:system_app:s0 tcontext=u:object_r:proc_stat:s0 tclass=file permissive=0

To add this SELinux rule temporarily, magiskpolicy can be used. To obtain the parameters required for magiskpolicy to add this SELinux rule, use the Linux tool audit2allow:

On the PC do:

First retrieve the current SELinux policy from the phone:

adb pull /sys/fs/selinux/policy

This command creates the file policy in the current directory on the PC.

With this file we can convert the message about the missing SELinux rule with audit2allow into the required SELinux rule:

echo "<log_message>" | audit2allow -p policy

Example:

[ OmniRom 14 Dev - xtrnaw7@t15g /tmp ] $ echo "us.atd.smmitest: type=1400 audit(0.0:153): avc: denied { open } for path="/proc/stat" dev="proc" ino=4026532098 scontext=u:r:system_app:s0 tcontext=u:object_r:proc_stat:s0 tclass=file permissive=0" | audit2allow -p policy #============= system_app ============== allow system_app proc_stat:file open;[ OmniRom 14 Dev - xtrnaw7@t15g /tmp ] $

The missing SELinux rule is the line beginning with "allow", which can be used as a parameter for magiskpolicy (as user root):

magiskpolicy --live "allow system_app proc_stat file open"
[Update 27.08.2024]

To check that the new rule is in place use this command:

magiskpolicy --print-rules | grep "allow system_app proc_stat file { open }"

The output should look like this:

ASUS_I006D:/ # magiskpolicy --print-rules | grep "allow system_app proc_stat file { open }" allow system_app proc_stat file { open }ASUS_I006D:/ #

That's it .

If more than one SELinux rule is missing, repeat the steps until all required SELinux policies for the application are present.

Note that the SELinux rules added using this method are only temporary and will not survive a reboot. To add these SELinux rules permanently, create a script in /data/adb/service.d with the magiskpolicy commands

Update 29.08.2024

Note that some SELinux rules, even if they appear to be active, are not allowed on Android and are therefore ignored.

If in doubt, ask for example ChatGPT a question like this:

"is this SELINux rule “allow untrusted_app_29 device sock_file write” allowed?"

(this is much faster than reading all the Android documentation ....)

Notes

see also Some hints for SELinux usage in Android

There is a small app called "SELinuxModeChanger" on F-Droid to change the SELinux status:

https://f-droid.org/repo/com.mrbimc.selinux_20171031.apk

That app also works on Android 14.

see Instructions to use the ASUS Hardware Testtool for an example usage of this apporach.

Don't forget to reactivate SELinux after testing by either reactivating it manually or restarting the phone.

How to add the ASUS Hardware Testtool as prebuild app for a Custom ROM

URL: https://xdaforums.com/t/how-to-add-the-asus-hardware-testtool-as-prebuild-app-for-a-custom-rom.4671918/

How to add the ASUS Hardware Testtool as prebuild app for a Custom ROM

In the post Instructions to use the ASUS Hardware Testtool I described how to add the ASUS Hardware Testtool to a CustomROM using Magisk.

For CustomROMs that you compile yourself, the ASUS Hardware Test Tool can also be added as a prebuild app as described below.

This post only contains the commands necessary to add the ASUS Hardware TestTool as prebuild app. For details about adding a prebuild app to a self compiled Custom ROM see How to add additional prebuild apks to a custom ROM and for detailed instructions to re-sign an apk file see How to re-sign an apk file .

In this example the CustomROM used is the OmniROM but the instructions should work for other CustomROMs also.

First copy the apks with the ASUS Hardware Testool and the ASUS calculator from the phone while the original ASUS Android OS is running on the phone:
# the directory may be different in your environment # cd /data/develop/android/SMMI_TEST adb pull /system/app/SMMI_TEST/SMMI_TEST.apk adb pull /system/priv-app/AsusCalculator/AsusCalculator.apk

Note:

The AsusCalculator is optional -- it's only required if you want to start the Hardware Tests via GUI on the phone

Now re-sign the apk file with the ASUS Hardware Testool using the certificate used to sign your self-compiled OS image:

# ---------------------------------------------------------------------------- # environment variables used in the commands below # APKSIGNER="/data/develop/android/android_sdk/build-tools/34.0.0/lib/apksigner.jar" REPO_DIR="/data/develop/android/OmniROM_14.0" DIR_WITH_KEYS="${REPO_DIR}/build/make/target/product/security/" ZIPALIGN="/data/develop/android/otatools/bin/zipalign" # ---------------------------------------------------------------------------- # re-sign both apks with the certificate used for your self compiled ROM # # ---------------------------------------------------------------------------- # re-sign the apk file for the Test app # zip -d SMMI_TEST.apk META-INF/CERT.SF META-INF/CERT.RSA ${ZIPALIGN} 4 SMMI_TEST.apk SMMI_TEST.apk.aligned mv SMMI_TEST.apk.aligned SMMI_TEST.apk # test the result of the zipalign (there should be no output from this command) # ${ZIPALIGN} -c 4 SMMI_TEST.apk # re-sign the apk with the certificate used for the new ROM # java -jar ${APKSIGNER} sign --key ${DIR_WITH_KEYS}/platform.pk8 --cert ${DIR_WITH_KEYS}/platform.x509.pem SMMI_TEST.apk # check the result keytool -printcert -jarfile SMMI_TEST.apk

Next re-sign the apk file with the ASUS Calculator using the certificate used to sign your self-compiled OS image:

# ---------------------------------------------------------------------------- # re-sign the apk file for the ASUS Calculator zip -d AsusCalculator.apk META-INF/AMAX.SF META-INF/AMAX.RSA ${ZIPALIGN} 4 AsusCalculator.apk AsusCalculator.apk.aligned mv AsusCalculator.apk.aligned AsusCalculator.apk # test the result of the zipalign (there should be no output from this command) # ${ZIPALIGN} -c 4 AsusCalculator.apk # re-sign the apk with the certificate used for the new ROM # java -jar ${APKSIGNER} sign --key ${DIR_WITH_KEYS}/platform.pk8 --cert ${DIR_WITH_KEYS}/platform.x509.pem AsusCalculator.apk # check the result # keytool -printcert -jarfile AsusCalculator.apk

Now create the config for the prebuild apps in the device tree with the source for the custom ROM:

# ---------------------------------------------------------------------------- # create the prebuild config for the ASUS Hardware Testool# cd ${REPO_DIR}/vendor/omni/prebuilt/app mkdir SMMI_TEST cd SMMI_TEST cp /data/develop/android/SMMI_TEST/SMMI_TEST.apk SMMI_TEST.apk cat >Android.mk <<-\EOT LOCAL_PATH:= $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := SMMI_TEST LOCAL_SRC_FILES := SMMI_TEST.apk LOCAL_MODULE_CLASS := APPS LOCAL_MODULE_TAGS := optional LOCAL_CERTIFICATE := PRESIGNED LOCAL_DEX_PREOPT := false LOCAL_PRIVILEGED_MODULE = true include $(BUILD_PREBUILT) EOT # ---------------------------------------------------------------------------- # create the prebuild config for the ASUS Calculator # cd ${REPO_DIR}/vendor/omni/prebuilt/app mkdir AsusCalculator cd AsusCalculator cp /data/develop/android/SMMI_TEST/AsusCalculator.apk . cat >Android.mk <<-\EOT LOCAL_PATH:= $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := AsusCalculator LOCAL_SRC_FILES := AsusCalculator.apk LOCAL_MODULE_CLASS := APPS LOCAL_MODULE_TAGS := optional LOCAL_CERTIFICATE := PRESIGNED LOCAL_DEX_PREOPT := false LOCAL_PRIVILEGED_MODULE = true include $(BUILD_PREBUILT) EOT # ----------------------------------------------------------------------------

Now add the packages to the file

${REPO_DIR}/vendor/omni/config/packages.mk
Use an editor and add these lines anywhere in the file BEFORE the last lines starting with "$("

PRODUCT_PACKAGES += \ SMMI_TEST \ AsusCalculator

# ---------------------------------------------------------------------------- Finally add the necessary SELinux policies for the ASUS Hardware Testtool:

echo "allow system_app proc_stat:file { ioctl read getattr lock map open watch watch_reads };" >${REPO_DIR}/device/asus/zenfone8/sepolicy/vendor/asus_hardware_testtool.te
# ----------------------------------------------------------------------------

Double check all files (especially the Android.mk files for leading or trailing whitespaces) and, if everything is fine, recreate the OS image using the standard commands and install the created OS image on the phone.

To start the Test App use either as user root in an adb shell:
# start the ASUS Hardware TestTool from within a shell # am start -n com.asus.atd.smmitest/com.asus.atd.smmitest.main.MAIN

or open the ASUS Calculator and enter

,12345+=

or (depending on your language settings)

.12345+=

How to add the ASUS Hardware Testtool as prebuild app for the StatiXOS ROM

URL: https://xdaforums.com/t/how-to-add-the-asus-hardware-testtool-as-prebuild-app-for-a-custom-rom.4671918/#post-89528935

How to add the ASUS Hardware Testtool as prebuild app for the StatiXOS ROM

I successfully added the ASUS Hardeware Testtool as prebuild app for the StatiXOS ROM for running on an ASUS Zenfone 8 using these instructions:

Optional replace the certificates used for the ROM image in the directory ${REPO_DIR}/build/make/target/product/security/ with your own certificates

Prepare the apk files for the ASUS Hardware Testtool and (optional) for the ASUS Calculator as described in the article How to add the ASUS Hardware Testtool as prebuild app for a Custom ROM

Then execute these commands on the PC (the commands are for a PC running the Linux OS):

# directory with the repository for the StatiXOS # REPO_DIR="/datapool001/develop/statix" # add the package for the ASUS Hardware Testtool as prebuild app # cd "${REPO_DIR}" cd vendor/statix-prebuilts/ mkdir SMMI_TEST cd SMMI_TEST # copy the apk file (the source directory with the apk file is most probably different in your environment) # cp /data/develop/android/SMMI_TEST/SMMI_TEST.apk SMMI_TEST.apk cat >Android.mk <<-\EOT LOCAL_PATH:= $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := SMMI_TEST LOCAL_SRC_FILES := SMMI_TEST.apk LOCAL_MODULE_CLASS := APPS LOCAL_MODULE_TAGS := optional LOCAL_CERTIFICATE := PRESIGNED LOCAL_DEX_PREOPT := false LOCAL_PRIVILEGED_MODULE = true include $(BUILD_PREBUILT) # add the package for the ASUS Calcuator as prebuild app # cd "${REPO_DIR}" cd vendor/statix-prebuilts/ mkdir AsusCalculator cd AsusCalculator # copy the apk file (the source directory with the apk file is most probably different in your environment) # cp /data/develop/android/SMMI_TEST/AsusCalculator.apk . cat >Android.mk <<-\EOT LOCAL_PATH:= $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := AsusCalculator LOCAL_SRC_FILES := AsusCalculator.apk LOCAL_MODULE_CLASS := APPS LOCAL_MODULE_TAGS := optional LOCAL_CERTIFICATE := PRESIGNED LOCAL_DEX_PREOPT := false LOCAL_PRIVILEGED_MODULE = true LOCAL_REPLACE_PREBUILT_APK_INSTALLED := $(LOCAL_PATH)/AsusCalculator.apk include $(BUILD_PREBUILT) EOT # add these lines to the file "vendor/statix/config/packages.mk": PRODUCT_PACKAGES += \     SMMI_TEST \     AsusCalculator # add the necessary SELinux policy for the ASUS Hardware Testtool # echo "allow system_app proc_stat:file { ioctl read getattr lock map open watch watch_reads };" >${REPO_DIR}device/asus/sake/sepolicy/vendor/asus_hardware_testtool.te Now rebuild the OS image . build/envsetup.sh time brunch statix_sake-ap1a-user

Note

sake in the parameter for the brunch command is the code name for the ASUS Zenfone 8.

The phones supported by StatiXOS are listed here: https://downloads.statixos.com/14-UPSIDEDOWNCAKE/ (You must know the codename for your phone to use that list)

ap1a in the parameter for the brunch command is the current Android release at time of this writing [26.05.2024] and will change in the future (check the current value in README for the StatiXOS: https://github.com/StatiXOS/android_manifest)

Notes

General infos about the StatixOS are here:    https://xdaforums.com/t/rom-upsidedowncake-sake-14-statixos-v7-5.4500497/

Build instructions for the StatiXOS are here:   https://github.com/StatiXOS/android_manifest

History of this entry

26.05.2024

initial release

14.07.2024

fixed typos (replaced apa1a with the correct value ap1a)

Some hints about EDL mode

URL: https://xdaforums.com/t/some-hints-about-edl-mode.4672703/

Some hints about EDL mode

This post contains some information about the EDL mode that I found out while testing the EDL mode for the ASUS Zenfone 8. I am not an expert in this field, so this article is probably neither complete nor error-free.

I only tested EDL mode using a PC running Linux (Fedora 39).

EDL mode is a special mode for phones using one of the Qualcomm CPUs.

"In this mode, the device identifies itself as Qualcomm HS-USB 9008 through USB. EDL is implemented by the PBL. Since the PBL is a ROM resident, EDL cannot be corrupted by software. The EDL mode itself implements the Qualcomm Sahara protocol, which accepts an OEM-digitally-signed programmer (an ELF binary in recent devices, MBN in older ones) over USB, that acts as an SBL. Modern such programmers implement the Firehose protocol, analyzed next." (copied from here : https://alephsecurity.com/vulns/aleph-2017028)

"EDL or Emergency DownLoad Mode is a special boot mode in Qualcomm Android devices that allows OEMs to force-flash firmware files. This special mode of operation is also commonly used by power users to unbrick their devices." EDL mode is not intended for the normal phone user. So if you don't know why you should use it, don't use it.
The necessary tools for Linux to access a phone in EDL mode can be downloaded from here:

https://github.com/bkerler/edl

Use the installation instructions for your Linux distribution on that page to install the EDL tools.

Or use the LiveCD with the tools available on the Website.

There are also instructions for installing the tools on a PC running the Windows "OS" (in the end it's "only" a python script) but I did not test them.

Another Windows tool to access a phone in EDL mode is available here: https://www.temblast.com/edl.htm

Some important things you might miss (at least I did ...).

SELinux must be disabled if active using the command:

sudo setenforce 0

The required udev rules for EDL must be in place, e.g.

cat /etc/udev/rules.d/51-edl.rules

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $ cat /etc/udev/rules.d/51-edl.rules # Qualcomm EDL SUBSYSTEMS=="usb", ATTRS{idVendor}=="05c6", ATTRS{idProduct}=="9008", MODE="0666", GROUP="plugdev" # Sony EDL SUBSYSTEMS=="usb", ATTRS{idVendor}=="0fce", ATTRS{idProduct}=="9dde", MODE="0666", GROUP="plugdev" SUBSYSTEMS=="usb", ATTRS{idVendor}=="0fce", ATTRS{idProduct}=="ade5", MODE="0666", GROUP="plugdev" # Qualcomm Memory Debug SUBSYSTEMS=="usb", ATTRS{idVendor}=="05c6", ATTRS{idProduct}=="9006", MODE="0666", GROUP="plugdev" # Qualcomm Memory Debug SUBSYSTEMS=="usb", ATTRS{idVendor}=="05c6", ATTRS{idProduct}=="900E", MODE="0666", GROUP="plugdev" # LG Memory Debug SUBSYSTEMS=="usb", ATTRS{idVendor}=="1004", ATTRS{idProduct}=="61a1", MODE="0666", GROUP="plugdev" # Sierra Wireless SUBSYSTEMS=="usb", ATTRS{idVendor}=="1199", ATTRS{idProduct}=="9071", MODE="0666", GROUP="plugdev" # ZTE SUBSYSTEMS=="usb", ATTRS{idVendor}=="19d2", ATTRS{idProduct}=="0076", MODE="0666", GROUP="plugdev" [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $

If this is not the case install the udev rules using these commands:

cd <edl_tools_dir> sudo cp Drivers/51-edl.rules /etc/udev/rules.d sudo udevadm control --reload sudo udevadm trigger

Afterwards, disconnect and reconnect the phone to use the new udev rules.

Note that the udev rules for EDL mode in the rules file that is part of the edl tools should be fine for all phones with a Qualcomm CPU.

To double check, compare the USB Vendor ID and Product ID of your phone while in EDL mode with the contents of the file /etc/udev/rules-d/51-edl.rules, example:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $ lsusb | grep QDL Bus 003 Device 037: ID 05c6:9008 Qualcomm, Inc. Gobi Wireless Modem (QDL mode) [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $ grep 05c6 /etc/udev/rules.d/51-edl.rules SUBSYSTEMS=="usb", ATTRS{idVendor}=="05c6", ATTRS{idProduct}=="9008", MODE="0666", GROUP="plugdev" SUBSYSTEMS=="usb", ATTRS{idVendor}=="05c6", ATTRS{idProduct}=="9006", MODE="0666", GROUP="plugdev" SUBSYSTEMS=="usb", ATTRS{idVendor}=="05c6", ATTRS{idProduct}=="900E", MODE="0666", GROUP="plugdev" [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $ To access the phone via EDL a Firehose loader for the phone is required. There are already some Firehose loader included in the EDL tools in the sub directories of the directory ./Loaders:[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $ ls Loaders/ amazon blackphone f10q GM hmd hydrogen letv lyf meizu mmx oneplus putyourloadersinhere.txt README.md smartisan TCL vivo zte asus_wingtech blackshark fancymaker haier HuaQin __init__.py LG megafone micromax nokia_foxconn ontim qmc samsung sonim tplink xiaomi blackberry cyanogen gionee hisense_agm huawei lenovo_motorola longcheer meitu microsoft nothing oppo qualcomm sharp sony unknown yulong_coolpad [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $

There is also a repository for Loaders available at https://github.com/bkerler/Loaders that you can check for a loader for your phone.

A Firehose loader for the ASUS Zenfone 8 is available here ASUS_Zenfone8_prog_firehose_ddr.elf and here is a zip file with other Firehose loader for the ASUS Zenfone 8: 02A firehose.zip. (all Firehose loader from hat archive work on my ASUS Zenfone 8)

A table with known Firehose loader is available here http://www.temblast.com/ref/loaders.htm.

To boot the phone into EDL mode, first boot the phone into the Android OS with working access via adb and then execute on the PC:
adb reboot edl

If rebooting into EDL works, the phone should only show a blank screen now and neither fastboot nor adb should detect a phone.

To check if the phone is in EDL mode, use lsusb, e.g:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $ lsusb .... Bus 003 Device 031: ID 05c6:9008 Qualcomm, Inc. Gobi Wireless Modem (QDL mode) ... [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $

The phone in EDL mode is the USB device in "QDL mode". If such a device is not present, the phone is not in EDL mode.

Before you can execute EDL commands, you must load the Firehose loader for your device using this command (replace prog_firehose_ddr.elf with the loader for your phone)

./edl --loader=/data/backup/ASUS_ZENFONE8/edl/prog_firehose_ddr.elf

This must be executed once as first command each time you enter the EDL mode.

Note

If you do not have a loader for your phone, you can test the automatic selection of the EDL script by running EDL without specifiying a loader, e.g:

./edl getstorageinfo

Output if edl does NOT find a loader for the phone

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $ ./edl getstorageinfo Qualcomm Sahara / Firehose Client V3.62 (c) B.Kerler 2018-2023. main - Trying with no loader given ... main - Waiting for the device ...... main - Hint: Press and hold vol up+dwn, connect usb. For some, only use vol up. main - Xiaomi: Press and hold vol dwn + pwr, in fastboot mode connect usb. Run "./fastpwn oem edl". main - Other: Run "adb reboot edl". .........main - Device detected :) sahara - Protocol version: 2, Version supported: 1 main - Mode detected: sahara sahara - Version 0x2 ------------------------ HWID: 0x001350e100290875 (MSM_ID:0x001350e1,OEM_ID:0x0029,MODEL_ID:0x0875) CPU detected: "lahaina" PK_HASH: 0x0e172e7799ab2674eacaee35e098e8b505ae768f7a2418ca653abecbe02198c3e3993a98b34fd8ecbb971c0c8b770e00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 Serial: 0x4dcc2414 sahara sahara - [LIB]: Couldn't find a loader for given hwid and pkhash (001350e100290875_0e172e7799ab2674_[FHPRG/ENPRG].bin) :(

Output if edl finds a loader for the phone

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $ ./edl getstorageinfo Qualcomm Sahara / Firehose Client V3.62 (c) B.Kerler 2018-2023. main - Trying with no loader given ... main - Waiting for the device main - Device detected :) main - Mode detected: sahara Traceback (most recent call last): File "/data/develop/android/edl/./edl", line 393, in <module> base.run() File "/data/develop/android/edl/./edl", line 295, in run version = conninfo["data"].version ^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: type object 'req' has no attribute 'version' [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $ ./edl getstorageinfo Qualcomm Sahara / Firehose Client V3.62 (c) B.Kerler 2018-2023. main - Trying with no loader given ... main - Waiting for the device ...... main - Hint: Press and hold vol up+dwn, connect usb. For some, only use vol up. main - Xiaomi: Press and hold vol dwn + pwr, in fastboot mode connect usb. Run "./fastpwn oem edl". main - Other: Run "adb reboot edl". .......main - Device detected :) sahara - Protocol version: 2, Version supported: 1 main - Mode detected: sahara sahara - Version 0x2 ------------------------ HWID: 0x001350e100290875 (MSM_ID:0x001350e1,OEM_ID:0x0029,MODEL_ID:0x0875) CPU detected: "lahaina" PK_HASH: 0x0e172e7799ab2674eacaee35e098e8b505ae768f7a2418ca653abecbe02198c3e3993a98b34fd8ecbb971c0c8b770e00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 Serial: 0x4dcc2414 sahara - Detected loader: /data/develop/android/edl/edlclient/../Loaders/asus_wingtech/001350e100290875_0e172e7799ab2674_fhrpg.bin sahara - Protocol version: 2, Version supported: 1 sahara - Uploading loader /data/develop/android/edl/edlclient/../Loaders/asus_wingtech/001350e100290875_0e172e7799ab2674_fhrpg.bin ... sahara - 64-Bit mode detected. sahara - Firehose mode detected, uploading... sahara - Loader successfully uploaded. main - Trying to connect to firehose loader ... firehose - INFO: Binary build date: Aug 13 2021 @ 23:00:54 firehose - INFO: Binary build date: Aug 13 2021 @ 23:00:54 firehose - INFO: Chip serial num: 1305224212 (0x4dcc2414) firehose - INFO: Supported Functions (15): firehose - INFO: program firehose - INFO: read firehose - INFO: nop firehose - INFO: patch firehose - INFO: configure firehose - INFO: setbootablestoragedrive firehose - INFO: erase firehose - INFO: power firehose - INFO: firmwarewrite firehose - INFO: getstorageinfo firehose - INFO: benchmark firehose - INFO: emmc firehose - INFO: ufs firehose - INFO: fixgpt firehose - INFO: getsha256digest firehose - INFO: End of supported functions 15 firehose_client firehose_client - [LIB]: No --memory option set, we assume "UFS" as default ..., if it fails, try using "--memory" with "UFS","NAND" or "spinor" instead ! firehose firehose - [LIB]: Couldn't detect MaxPayloadSizeFromTargetinBytes firehose firehose - [LIB]: Couldn't detect TargetName firehose - TargetName=Unknown firehose - MemoryName=UFS firehose - Version=1 firehose - Trying to read first storage sector... firehose - Running configure... firehose - Storage report: firehose - total_blocks:30625792 firehose - block_size:4096 firehose - page_size:4096 firehose - num_physical:7 firehose - manufacturer_id:462 firehose - serial_num:1297306958 firehose - fw_version:100 firehose - mem_type:UFS firehose - prod_name:KLUDG4UHDC-B0E1 firehose_client - Supported functions: ----------------- program,read,nop,patch,configure,setbootablestoragedrive,erase,power,firmwarewrite,getstorageinfo,benchmark,emmc,ufs,fixgpt,getsha256digest firehose - GetStorageInfo: -------------------- firehose - INFO: Calling handler for getstorageinfo firehose - INFO: Device Total Logical Blocks: 0x1d35000 firehose - INFO: Device Block Size in Bytes: 0x1000 firehose - INFO: Device Total Physical Partitions: 0x7 firehose - INFO: Device Manufacturer ID: 0x1ce firehose - INFO: Device Serial Number: 0x4d53554e firehose - INFO: {"storage_info": {"total_blocks":30625792, "block_size":4096, "page_size":4096, "num_physical":7, "manufacturer_id":462, "serial_num":1297306958, "fw_version":"100","mem_type":"UFS","prod_name":"KLUDG4UHDC-B0E1"}} firehose - INFO: UFS fInitialized: 0x1 firehose - INFO: UFS Current LUN Number: = 0x0 firehose - INFO: UFS Total Active LU: 0x7 firehose - INFO: UFS wManufacturerID: 0x1ce firehose - INFO: UFS Boot Partition Enabled: 0x1 firehose - INFO: UFS Raw Device Capacity: = 0xee64000 firehose - INFO: UFS Min Block Size: 0x8 firehose - INFO: UFS Erase Block Size: 0x2000 firehose - INFO: UFS Allocation Unit Size: 0x1 firehose - INFO: UFS RPMB ReadWrite Size: = 0x40 firehose - INFO: UFS Number of Allocation Uint for This LU: 0x74d4 firehose - INFO: UFS Logical Block Size: 0xc firehose - INFO: UFS Provisioning Type: 0x2 firehose - INFO: UFS LU Write Protect: 0x0 firehose - INFO: UFS Boot LUN ID: = 0x0 firehose - INFO: UFS Memory Type: 0x0 firehose - INFO: UFS LU Total Blocks: 0x1d35000 firehose - INFO: UFS Supported Memory Types: 0x800f firehose - INFO: UFS dEnhanced1MaxNAllocU: 0x7732 firehose - INFO: UFS wEnhanced1CapAdjFac: 0x300 firehose - INFO: UFS dEnhanced2MaxNAllocU: = 0x0 firehose - INFO: UFS wEnhanced2CapAdjFac: 0x0 firehose - INFO: UFS dEnhanced3MaxNAllocU: 0x0 firehose - INFO: UFS wEnhanced3CapAdjFac: 0x0 firehose - INFO: UFS dEnhanced4MaxNAllocU: 0x0 firehose - INFO: UFS wEnhanced4CapAdjFac: 0x0 firehose - INFO: UFS LUN Enable Bitmask: 0x7f firehose - INFO: UFS Logical Block Count: 0x1d35000 firehose - INFO: UFS bConfigDescrLock: 0x0 firehose - INFO: UFS iManufacturerName String Index: 0x0 firehose - INFO: UFS iProductName String Index: 0x1 firehose - INFO: UFS iSerialNumber String Index: 0x2 firehose - INFO: UFS iOemID String Index: 0x3 firehose - INFO: UFS Inquiry Command Output: SAMSUNG KLUDG4UHDC-B0E1 0100 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $

To test, if the access via EDL is working, use one of the EDL commands to read data from the phone, e.g

./edl getactiveslot

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $ ./edl getactiveslot Qualcomm Sahara / Firehose Client V3.62 (c) B.Kerler 2018-2023. main - Trying with no loader given ... main - Waiting for the device main - Device detected :) main - Mode detected: firehose Current active slot: b [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $

See the documentation for the edl tool for what you can do in EDL mode.

If everything is done, exit the EDL mode using the command:

./edl reset

or, if that does not work :

Press the buttons VolumeUp and VolumeDown and Power together for at least 10 seconds.

Trouble Shooting

"adb reboot edl" does not work with all OS available for the ASUS Zenfone 8:

ROM	"adb reboot edl" works?	Comment
ASUS Android 13	yes
OmniROM 13	yes
OmniROM 14	yes
LineageOS 19	no	the phone boots into the Android OS after executing "adb reboot edl" Booting into EDL mode via adb does also not work if the phone is booted into the recovery Booting via adb into the EDL mode works if the phone is booted from the TWRP image file.
LineageOS 20	no	the phone boots into the Android OS after executing "adb reboot edl" Booting into EDL mode via adb does also not work if the phone is booted into the recovery Booting via adb into the EDL mode also does not work if the phone is booted from the TWRP image file created for the LineageOS (see here)
LineageOS 21 (unofficial build!)	yes	OS image downloaded from here: https://github.com/mikooomich/android_device_asus_sake/releases
/e/	no	the phone boots into the Android OS after executing "adb reboot edl" Booting into EDL mode via adb does also not work if the phone is booted into the recovery Booting via adb into the EDL mode also does not work if the phone is booted from the TWRP image file created for the StatiXOS (see here)
StatiXOS	no	the phone boots into the Android OS after executing "adb reboot edl" Booting into EDL mode via adb does also not work if the phone is booted into the recovery Booting via adb into the EDL mode also does not work if the phone is booted from the TWRP image file created for the StatiXOS (see here)
LMODroid	no	the phone boots into the Android OS after executing "adb reboot edl"
TWRP 3.7.1_12-0	yes

The "special" TWRP images created for LineageOS and the other OS are build using the ramdisk from TWRP and the kernel from the original OS. So I guess booting into EDL mode is a feature of the kernel that is not enabled in these kernel.

Physically booting the ASUS Zenfone 8 into the EDL mode

If there is an OS installed on the phone that does not support booting into the EDL mode via adb command, the phone must be booted physically into the EDL mode

The instructions how to do that are printed by the edl script:

main - Hint: Press and hold vol up+dwn, connect usb. For some, only use vol up.
With the help of aunt google I found these more detailed working instructions:

power off the phone using the PowerOff entry in the Bootloader menu
wait until the phone is really powered off
disconnect the phone from the PC
Hold volume up and volume down at the same time
start edl on the PC
connect the phone to the PC while holding down both buttons
release both buttons when the screen is black again (after at least 5 seconds)

But this works only partially on the ASUS Zenfone 8 :

The phone is now visible with a different vendor id and a different product it:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ lsusb | grep Qual Bus 003 Device 012: ID 05c6:f000 Qualcomm, Inc. TA-1004 [Nokia 8] [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

These USB IDs are not yet configured in the file with the udev rules for the EDL mode /etc/udev/rules.d/51-edl.rules :

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ grep 05c6 /etc/udev/rules.d/51-edl.rules SUBSYSTEMS=="usb", ATTRS{idVendor}=="05c6", ATTRS{idProduct}=="9008", MODE="0666", GROUP="plugdev" SUBSYSTEMS=="usb", ATTRS{idVendor}=="05c6", ATTRS{idProduct}=="9006", MODE="0666", GROUP="plugdev" SUBSYSTEMS=="usb", ATTRS{idVendor}=="05c6", ATTRS{idProduct}=="900E", MODE="0666", GROUP="plugdev" [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Therefor edl started as non-root user can not detect the phone.

To fix this issue, add the new USB ids to the file /etc/udev/rules.d/51-edl.rules :
cp /etc/udev/rules.d/51-edl.rules $HOME/51-edl.rules.org echo 'SUBSYSTEMS=="usb", ATTRS{idVendor}=="05c6", ATTRS{idProduct}=="f000", MODE="0666", GROUP="plugdev"' | sudo tee -a /etc/udev/rules.d/51-edl.rules [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ grep 05c6 /etc/udev/rules.d/51-edl.rules SUBSYSTEMS=="usb", ATTRS{idVendor}=="05c6", ATTRS{idProduct}=="9008", MODE="0666", GROUP="plugdev" SUBSYSTEMS=="usb", ATTRS{idVendor}=="05c6", ATTRS{idProduct}=="9006", MODE="0666", GROUP="plugdev" SUBSYSTEMS=="usb", ATTRS{idVendor}=="05c6", ATTRS{idProduct}=="900E", MODE="0666", GROUP="plugdev" SUBSYSTEMS=="usb", ATTRS{idVendor}=="05c6", ATTRS{idProduct}=="f000", MODE="0666", GROUP="plugdev" [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ sudo udevadm control --reload sudo udevadm trigger
But even with the correct udev rules the phone is not detected by the EDL tool. This is not an EDL mode. Most probably ASUS disabled switching to the EDL mode using the physical buttons in the firmware.

Conclusion:

The only way to boot an ASUS Zenfone 8 with one of the crippled operating systems into EDL mode is to use the special USB cable to boot into EDL mode (see for example here).

Update 11.06.2024

ASUS Zenfone 8 will also not boot into EDL mode when using a Deep Flash cable

If an error occurs when using the edl tool, first check whether the error is already mentioned in a problem in the repository's problem list: : https://github.com/bkerler/edl/issues

The python script edl writes debug messages if executed with the parameter --debugmode.

The parameter --debugmode only works if the directory logs exists in the current working directory and the current user can write to this directory. The debug messages are then in the file ./logs/log.txt.

The ASUS Zenfone 8 can not be booted into the EDL mode via fastboot (at least in my environment with Fedora 39)

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $ ./edl reset Qualcomm Sahara / Firehose Client V3.62 (c) B.Kerler 2018-2023. main - Trying with no loader given ... main - Waiting for the device main - Device detected :) main - Mode detected: firehose DeviceClass - USBError(19, 'No such device (it may have been disconnected)') [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $
The error message after doing a edl reset seems to be normal and not an error.

The reason of the error message
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $ ./edl --loader=./prog_firehose_ddr.elf printgpt Qualcomm Sahara / Firehose Client V3.62 (c) B.Kerler 2018-2023. main - Using loader ./prog_firehose_ddr.elf ... main - Waiting for the device DeviceClass DeviceClass - [LIB]: Couldn't get device configuration. .DeviceClass

was in my environment the missing udev rules file.

It seems, that not all firehole loader support all commands:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $ ./edl pbl ./primary_boot_loader Qualcomm Sahara / Firehose Client V3.62 (c) B.Kerler 2018-2023. main - Trying with no loader given ... main - Waiting for the device main - Device detected :) main - Mode detected: firehose firehose_client firehose_client - [LIB]: Peek command isn't supported by edl loader [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $

The command

./edl getstorageinfo

prints the list of supported functions, e.g.:

/edl --loader=asus_firehose/firehose/prog_firehose_ddr.elf getstorageinfo

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $ ./edl --loader=asus_firehose/firehose/prog_firehose_ddr.elf getstorageinfo Qualcomm Sahara / Firehose Client V3.62 (c) B.Kerler 2018-2023. main - Using loader asus_firehose/firehose/prog_firehose_ddr.elf ... main - Waiting for the device ...... main - Hint: Press and hold vol up+dwn, connect usb. For some, only use vol up. main - Xiaomi: Press and hold vol dwn + pwr, in fastboot mode connect usb. Run "./fastpwn oem edl". main - Other: Run "adb reboot edl". ......main - Device detected :) sahara - Protocol version: 2, Version supported: 1 main - Mode detected: sahara sahara - Version 0x2 ------------------------ HWID: 0x001350e100290875 (MSM_ID:0x001350e1,OEM_ID:0x0029,MODEL_ID:0x0875) CPU detected: "lahaina" PK_HASH: 0x0e172e7799ab2674eacaee35e098e8b505ae768f7a2418ca653abecbe02198c3e3993a98b34fd8ecbb971c0c8b770e00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 Serial: 0x4dcc2414 sahara - Protocol version: 2, Version supported: 1 sahara - Uploading loader asus_firehose/firehose/prog_firehose_ddr.elf ... sahara - 64-Bit mode detected. sahara - Firehose mode detected, uploading... sahara - Loader successfully uploaded. main - Trying to connect to firehose loader ... firehose - INFO: Binary build date: Aug 13 2021 @ 23:00:54 firehose - INFO: Binary build date: Aug 13 2021 @ 23:00:54 firehose - INFO: Chip serial num: 1305224212 (0x4dcc2414) firehose - INFO: Supported Functions (15): firehose - INFO: program firehose - INFO: read firehose - INFO: nop firehose - INFO: patch firehose - INFO: configure firehose - INFO: setbootablestoragedrive firehose - INFO: erase firehose - INFO: power firehose - INFO: firmwarewrite firehose - INFO: getstorageinfo firehose - INFO: benchmark firehose - INFO: emmc firehose - INFO: ufs firehose - INFO: fixgpt firehose - INFO: getsha256digest firehose - INFO: End of supported functions 15 firehose_client ...

Running edl with sudo only works if the tool has been installed and configured as root user (and this is not necessary and explicitly NOT recommended)

There are various reasons to get an error message like this:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ]
        $ ./edl getstorageinfo

        Qualcomm Sahara / Firehose Client V3.62 (c) B.Kerler 2018-2023.

        main - Trying with no loader given ...

        main - Waiting for the device

        main - Device detected :)

        main - Mode detected: sahara

        Traceback (most recent call last):

        File "/data/develop/android/edl/./edl", line 393, in
        <module>

        base.run()

        File "/data/develop/android/edl/./edl", line 295, in run

        version = conninfo["data"].version

        ^^^^^^^^^^^^^^^^^^^^^^^^

        AttributeError: type object 'req' has no attribute 'version'

        [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/edl ] $

To fix it:

- use an USB 2.0 port and not an USB 3.x port

- try the access using a different USB cable

- in my environment these instructions work:

execute the commands to access the phone via EDL in this order:

1. boot the phone into the Android OS with enabled access via adb

2. open a terminal and start edl :

./edl
        --loader=/data/backup/ASUS_ZENFONE8/edl/prog_firehose_ddr.elf

(the script then complains that no phone was found in edl mode, but it waits until it finds one)

3. open another terminal and reboot the phone into the edl mode:

adb reboot edl

History of this entry

20.05.2024

initial release

How to boot the ASUS Zenfone 8 into the bootloader using the physical buttons

URL: not yet published

How to boot the phone into the bootloader using the physical buttons

If an ASUS Zenfone 8 is no longer "usable", e.g. no longer boots, is stuck in a boot loop, etc., you can use the physical buttons to restart the phone in the bootloader.

Power-off the phone using this approach : Press Volume Up, Volume Down, and the Power button at the same time for at least 10 seconds
when the phone is really powered off: Press Volume Up , Volume Down, and the Power button at the same time until the bootloader prompt appaers
now release the Power button but not the Volume Up and Volume Down buttons
next release the Volume Up and Volume Down button

Now the phone should be in the bootloader.

If the OS on the phone is corrupted and it's not possible to install a new one, you can always install a raw image of the ASUS Android for the ASUS Zenfone 8 see here for instructions https://xdaforums.com/t/full-recover-to-stock-if-things-went-really-bad.4337467/
Note that a raw image can also be installed on phones with locked boot loader.

Unfortunately, ASUS has removed the raw images from their website, but some of the raw images are still available for download on other sites; a list can be found in this forum post: https://xdaforums.com/t/zenfone-8-firmware-ota-collection.4620171/

The installation of a raw image destroys all data on the phone, but you can continue to use the phone by installing an official Android from ASUS again (or a CustomROM if the bootloader of the phone is locked).

Note that this approach also works if the phone is in the ramdump after a crash

How to reset the USB port used to connect an Android phone

URL: https://xdaforums.com/t/how-to-reset-the-usb-port-used-to-connect-an-android-phone.4672845/

How to reset the USB port used to connect an Android phone

During the development of my scripts to automatically install and configure the phone, the adb connection between the phone and the PC was often lost after restarting the phone for a reason unknown to me.
A workaround to re-establish the connection is to disconnect and reconnect the USB cable - but of course this is not a solution for an automatic installation of the phone.

That's why I was looking for a solution to solve this problem.

I finally found a solution: Resetting the USB port through which the phone is connected to the PC restored the adb connection.

So I wrote a small shell script for Linux that performs the reset:

reset_usb_port_for_phone.sh

The usage of the script is:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./reset_usb_port_for_phone.sh -h reset_usb_port_for_phone.sh v1.0.0 Usage: reset_usb_port_for_phone [serial_number_of_the_phone] [force] [noreset|view] Parameter: serial_number_of_the_phone the serial number of the phone for which the USB port should be reset the default is the serial number hardcoded in the script force reset the USB port even if it is not necessary noreset|view only print the current status; do NOT reset the USB port Using this parameter the script uses these return codes: 0 - there is a working adb connection to the phone 1 - the phone is in the bootloader 2 - the phone is in the fastbootd 9 - no phone with the given serial number found >100 - an error occured [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

The script uses the serial number of the phone to select the correct phone.

If the script is called without a parameter, it uses the hardcoded default serial number:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ grep ^DEFAULT_SERIAL_NUMBER ./reset_usb_port_for_phone.sh -h DEFAULT_SERIAL_NUMBER="M6AIB760D0939LX" [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

(You should of course change this value if you use the script)

The script uses a little program called usbreset to reset the USB port

You can either download the program from here, or download the source code from https://askubuntu.com/questions/645/how-do-you-reset-a-usb-device-from-the-command-line and compile it yourself:

A simple

cc usbreset.c -o usbreset

is sufficient to compile the source code; that should be possible on any Linux with an installed C compiler. The binary on my website was compiled in Fedora 39.

The binary usbreset must be available via PATH for the script.

reset_usb_port_for_phone.sh first checks whether the phone can be reached via fastboot or adb: If so, the reset is not necessary and is not carried out (use the parameter force to do a reset of the USB port anyway).

Example output of the script:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/test2 ] $ ./reset_usb_port_for_phone.sh reset_usb_port_for_phone.sh v1.0.1 Resetting the USB port used for the phone whith the serial number "M6AIB760D0939LX" if necessary ... Determine the USB port used for the phone ... The USB port for the phone with the s/n "M6AIB760D0939LX" is "/dev/bus/usb/003/125" (ASUSTek Computer, Inc. Zenfone 8) The phone with the serial number "M6AIB760D0939LX" is NOT available via adb Now resetting the USB port "/dev/bus/usb/003/125" used for the phone ... Resetting USB device /dev/bus/usb/003/125 Reset successful The status of the phone is now: M6AIB760D0939LX device Successfully resetting the USB port for the phone [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/test2 ] $

Output of the script if the phone is in the bootloader (fastboot mode)

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./reset_usb_port_for_phone.sh ; echo $? reset_usb_port_for_phone.sh v1.0.1 Resetting the USB port used for the phone whith the serial number "M6AIB760D0939LX" if necessary ... Determine the USB port used for the phone ... The USB port for the phone with the s/n "M6AIB760D0939LX" is "/dev/bus/usb/003/074" (ASUSTek Computer, Inc. Transformer Pad Infinity TF700 (Fastboot)) The phone with the serial number "M6AIB760D0939LX" is available via fastboot - M6AIB760D0939LX fastboot - No USB reset necessary 0 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Output of the script if the phone is in fastbootd mode

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./reset_usb_port_for_phone.sh ; echo $? reset_usb_port_for_phone.sh v1.0.1 Resetting the USB port used for the phone whith the serial number "M6AIB760D0939LX" if necessary ... Determine the USB port used for the phone ... The USB port for the phone with the s/n "M6AIB760D0939LX" is "/dev/bus/usb/003/083" (ASUSTek Computer, Inc. Transformer Pad Infinity TF700 (Fastboot)) The phone with the serial number "M6AIB760D0939LX" is available via fastboot - M6AIB760D0939LX fastbootd - No USB reset necessary 0 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Output of the script if there is a working adb connection

# LineageOS 21 recovery with enabled adb
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./reset_usb_port_for_phone.sh ; echo $? reset_usb_port_for_phone.sh v1.0.1 Resetting the USB port used for the phone whith the serial number "M6AIB760D0939LX" if necessary ... Determine the USB port used for the phone ... The USB port for the phone with the s/n "M6AIB760D0939LX" is "/dev/bus/usb/003/076" (ASUSTek Computer, Inc. ASUS_I006D) The phone with the serial number "M6AIB760D0939LX" is available via adb; the status is M6AIB760D0939LX    recovery No USB reset necessary 0 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

# LineageOS 21 recovery with not enabled adb

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./reset_usb_port_for_phone.sh ; echo $? reset_usb_port_for_phone.sh v1.0.1 Resetting the USB port used for the phone whith the serial number "M6AIB760D0939LX" if necessary ... Determine the USB port used for the phone ... The USB port for the phone with the s/n "M6AIB760D0939LX" is "/dev/bus/usb/003/075" (ASUSTek Computer, Inc. ASUS_I006D) The phone with the serial number "M6AIB760D0939LX" is available via adb; the status is M6AIB760D0939LX    unauthorized No USB reset necessary 0 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

# LineageOS 21 with enabled adb

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./reset_usb_port_for_phone.sh ; echo $? reset_usb_port_for_phone.sh v1.0.1 Resetting the USB port used for the phone whith the serial number "M6AIB760D0939LX" if necessary ... Determine the USB port used for the phone ... The USB port for the phone with the s/n "M6AIB760D0939LX" is "/dev/bus/usb/003/077" (ASUSTek Computer, Inc. Zenfone 8) The phone with the serial number "M6AIB760D0939LX" is available via adb; the status is M6AIB760D0939LX    device No USB reset necessary 0 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

# Lineage OS 21 with disabled adb

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./reset_usb_port_for_phone.sh ; echo $? reset_usb_port_for_phone.sh v1.0.1 Resetting the USB port used for the phone whith the serial number "M6AIB760D0939LX" if necessary ... Determine the USB port used for the phone ... The USB port for the phone with the s/n "M6AIB760D0939LX" is "/dev/bus/usb/003/078" (ASUSTek Computer, Inc. Zenfone GO (ZB500KL) (MTP mode)) Now resetting the USB port "/dev/bus/usb/003/078" used for the phone ... Resetting USB device /dev/bus/usb/003/078 Reset successful The status of the phone is now: Successfully resetting the USB port for the phone 0 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ]

# LineageOS 21 with enabled adb but missing authorisation

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./reset_usb_port_for_phone.sh ; echo $? reset_usb_port_for_phone.sh v1.0.1 Resetting the USB port used for the phone whith the serial number "M6AIB760D0939LX" if necessary ... Determine the USB port used for the phone ... The USB port for the phone with the s/n "M6AIB760D0939LX" is "/dev/bus/usb/003/079" (ASUSTek Computer, Inc. Zenfone 8) The phone with the serial number "M6AIB760D0939LX" is available via adb; the status is M6AIB760D0939LX    unauthorized No USB reset necessary 0 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

Output of the script if the phone is neither via adb nor via fastboot accessable

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ ../scripts_on_linux/reset_usb_port_for_phone.sh M6AIB760D0939LY view; echo $? reset_usb_port_for_phone.sh v1.0.1 Resetting the USB port used for the phone whith the serial number "M6AIB760D0939LY" if necessary ... Determine the USB port used for the phone ... ERROR: No phone with the serial number "M6AIB760D0939LY" found in the list of attached USB devices 9 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

Notes

Note that the USB port used for the connection to the phone is dynamic and changes for example after rebooting the phone. In addition, the device identifier for the USB device also depends on the booted OS on the phone

Examples

When the phone is booted into the LMODroid with enabled adb:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./reset_usb_port_for_phone.sh view ; echo $? reset_usb_port_for_phone.sh v1.0.0 Resetting the USB port used for the phone whith the serial number "M6AIB760D0939LX" if necessary ... Determine the USB port used for the phone ... The USB port for the phone with the s/n "M6AIB760D0939LX" is "/dev/bus/usb/003/079" (ASUSTek Computer, Inc. Zenfone 8) The phone with the serial number "M6AIB760D0939LX" is available via adb; the status is M6AIB760D0939LX    device 0 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

After booting the phone into the bootloader:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ adb reboot bootloader [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./reset_usb_port_for_phone.sh view ; echo $? reset_usb_port_for_phone.sh v1.0.0 Resetting the USB port used for the phone whith the serial number "M6AIB760D0939LX" if necessary ... Determine the USB port used for the phone ... The USB port for the phone with the s/n "M6AIB760D0939LX" is "/dev/bus/usb/003/080" (ASUSTek Computer, Inc. Transformer Pad Infinity TF700 (Fastboot)) The phone with the serial number "M6AIB760D0939LX" is available via fastboot - M6AIB760D0939LX    fastboot - 1 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

After booting the phone into the LMODroid recovery without enabling adb:
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./reset_usb_port_for_phone.sh view ; echo $? reset_usb_port_for_phone.sh v1.0.0 Resetting the USB port used for the phone whith the serial number "M6AIB760D0939LX" if necessary ... Determine the USB port used for the phone ... The USB port for the phone with the s/n "M6AIB760D0939LX" is "/dev/bus/usb/003/081" (ASUSTek Computer, Inc. ASUS_I006D) The phone with the serial number "M6AIB760D0939LX" is available via adb; the status is M6AIB760D0939LX    unauthorized 0 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

After enabling adb in the LMODroid recovery:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $ ./reset_usb_port_for_phone.sh view ; echo $? reset_usb_port_for_phone.sh v1.0.0 Resetting the USB port used for the phone whith the serial number "M6AIB760D0939LX" if necessary ... Determine the USB port used for the phone ... The USB port for the phone with the s/n "M6AIB760D0939LX" is "/dev/bus/usb/003/082" (ASUSTek Computer, Inc. ASUS_I006D) The phone with the serial number "M6AIB760D0939LX" is available via adb; the status is M6AIB760D0939LX    recovery 0 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts_on_linux ] $

When the phone is booted into the OmniROM with enabled adb:
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ ../scripts_on_linux/reset_usb_port_for_phone.sh view ; echo ? reset_usb_port_for_phone.sh v1.0.1 Resetting the USB port used for the phone whith the serial number "M6AIB760D0939LX" if necessary ... Determine the USB port used for the phone ... The USB port for the phone with the s/n "M6AIB760D0939LX" is "/dev/bus/usb/003/011" (ASUSTek Computer, Inc. LAHAINA-MTP _SN:4DCC2414) The phone with the serial number "M6AIB760D0939LX" is available via adb; the status is M6AIB760D0939LX    device ? [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

When the phone is in the TWRP installed on the phone running the OmniROM OS:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $ ../scripts_on_linux/reset_usb_port_for_phone.sh view ; echo ? reset_usb_port_for_phone.sh v1.0.1 Resetting the USB port used for the phone whith the serial number "M6AIB760D0939LX" if necessary ... Determine the USB port used for the phone ... The USB port for the phone with the s/n "M6AIB760D0939LX" is "/dev/bus/usb/003/013" (ASUSTek Computer, Inc. Zenfone GO (ZB500KL) (Debug, MTP mode)) The phone with the serial number "M6AIB760D0939LX" is available via adb; the status is M6AIB760D0939LX    recovery ? [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/OmniROM_14.0 ] $

How to change files in the boot image using Magisk if there is no ramdisk in the boot partition

URL: https://xdaforums.com/t/how-to-change-files-in-the-boot-image-using-magisk-if-there-is-no-ramdisk-in-the-boot-partition.4678017/

How to change files in the boot image using Magisk if there is no ramdisk in the boot partition

In this post How to change files in the boot image using Magisk I describe how to change files in the boot image for an Android OS using the Magisk binary magiskboot.

The instructions in this HowTo assume that the boot partition on the phone contains the kernel and the initial ramdisk.

But since Android 10 some Android ROMs use system-as-root configurations. In this configuration the RAM disk is part of the system partition -- see https://source.android.com/docs/core/architecture/partitions/system-as-root#sar-partitioning for details and see also this thread https://xdaforums.com/t/question-about-boot-img-and-ramdisk-cpio.4584293/.

An example for this configuration is the Stock Android 14 OS for the Google Pixel 6a (the installation image for the Google Pixel 6a was copied from this website https://developers.google.com/android/images)

The contents of the boot partition for the Stock Android 14 OS for the Google Pixel 6a looks like this:

[ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/test/bluejay-ap2a.240605.024/boot_img ] $ magiskboot unpack -h ../boot.img Parsing boot image: [../boot.img] HEADER_VER [4] KERNEL_SZ [24967056] RAMDISK_SZ [0] PAGESIZE [4096] CMDLINE [] KERNEL_FMT [lz4_legacy] VBMETA [ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/test/bluejay-ap2a.240605.024/boot_img ] $ ls -ltr total 29130 -rw-rw-rw-. 1 xtrnaw7 xtrnaw7 9 Jun 19 16:48 header -rw-r--r--. 1 xtrnaw7 xtrnaw7 51685888 Jun 19 16:48 kernel [ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/test/bluejay-ap2a.240605.024/boot_img ] $

-> There is only the kernel in the boot partition.

Therefor you can not change the files in the ramdisk by unpacking, changing, and repacking the ramdisk on the boot partition anymore.

But changing or adding files with Magisk's overlay feature also works for these ROMs. This is because Magisk creates a small ramdisk in the boot partition during installation and also changes the Android boot options so that the ramdisk on the boot partition is used if necessary.

Notes:

The contents of the ramdisk in the boot partition created by Magisk with the files necessary for Magisk are:

sake:/data/develop/boot_img_with_magisk/ramdisk # ls -ld $( find . )drwxr-xr-x 4 root root 3452 2024-06-20 16:32 . d--------- 2 root root 3452 2024-06-20 16:32 ./.backup ---------- 1 root root 128 1970-01-01 01:00 ./.backup/.magisk -rwxr-x--- 1 root root 680904 1970-01-01 01:00 ./init drwxr-x--- 3 root root 3452 2024-06-20 16:32 ./overlay.d drwxr-x--- 2 root root 3452 2024-06-20 16:32 ./overlay.d/sbin -rw-r--r-- 1 root root 102184 1970-01-01 01:00 ./overlay.d/sbin/magisk32.xz -rw-r--r-- 1 root root 117168 1970-01-01 01:00 ./overlay.d/sbin/magisk64.xz -rw-r--r-- 1 root root 25964 1970-01-01 01:00 ./overlay.d/sbin/stub.xz sake:/data/develop/boot_img_with_magisk/ramdisk #

Additional files for the boot image added via the Magisk overlay feature are stored in the cpio file for this ramdisk (see How to run a script at shutdown for an example - you can use the instructions from that post without changes also for phones running a ROM without a ramdisk on the boot partition).

See How to add files to the ramdisk used for the recovery boot for instructions to add additional files to the ramdisk used for recovery boots.

History of this entry

23.06.2024

initial release

How to add files to the ramdisk used for the recovery boot

URL: https://xdaforums.com/t/how-to-add-files-to-the-ramdisk-used-for-the-recovery-boot.4678287/

How to add files to the ramdisk used for the recovery boot

IMHO one drawback of the recoveries used for LineageOS-based ROMs is that some important tools are missing in the ramdisk for the recovery and that adb is not enabled by default in the recoveries of the user builds.
This post contains instructions to fix these issues without recreating the ROM image.

The ROMs /e/ 2.x, LMODroid 4.2, LineageOS 20, and LineageOS 21 for the ASUS Zenfone 8 use the ramdisk on the partition vendor_boot when they are booted in recovery mode (tested only with images for the ASUS Zenfone 8 but I assume this is true for other ROMs without a ramdisk in the boot partition).

To add files to the ramdisk that is used for the recovery mode in one of the ROMs listed above, proceed as follows.

Either install Magisk or enable root access via adb in the Developer settings if this setting is available in the ROM.

If Magisk is not installed on the phone, extract the executable magiskboot for the CPU in your phone from the Magisk apk file and copy it to the phone.
See here for instructions how to do that, e.g. to extract the executable for Android phones with an ARM 64 Bit CPU do:

unzip -p Magisk-v27.0.apk lib/arm64-v8a/libmagiskboot.so >magiskboot && chmod 755 magiskboot

Note that Magisk is only required to change the ramdisk on the vendor_boot partition - it is not mandatory to install Magisk on the phone.

Open an adb shell and issue as user root:

# change the path to magiskboot in the variable MAGISKBOOT according to your environment
#
MAGISKBOOT="/data/adb/magisk/magiskboot" # get the current boot slot # CUR_SLOT=$( getprop ro.boot.slot_suffix )# create the working directory # mkdir -p /data/develop/vendor_boot${CUR_SLOT} cd /data/develop/vendor_boot${CUR_SLOT} # extract and unpack the partition vendor_boot # dd if=/dev/block/by-name/vendor_boot${CUR_SLOT} of=./vendor_boot${CUR_SLOT}.img mkdir image cd image ${MAGISKBOOT} unpack -h ../vendor_boot${CUR_SLOT}.img# unpack the ramdisk from the vendor_boot partition # mkdir ramdisk cd ramdisk ${MAGISKBOOT} decompress ../ramdisk.cpio - | cpio -idm

(remember the compression format printed by magiskboot - it is required later to compress the cpio file with the ramdisk again. Example: Detected format: [lz4_legacy] )

The contents of the directory with the uncompressed ramdisk.cpio file should look like this:

ramdisk file contents

ASUS_I006D:/data/develop/vendor_boot_a/image/ramdisk # ls -ltr total 1696 -rw-r--r-- 1 root root 4306 1970-01-01 01:00 vendor_service_contexts -rw-r--r-- 1 root root 24715 1970-01-01 01:00 vendor_property_contexts -rw-r--r-- 1 root root 101389 1970-01-01 01:00 vendor_file_contexts drwxr-xr-x 2 root root 3488 1970-01-01 01:00 vendor drwxr-xr-x 2 root root 3488 1970-01-01 01:00 tmp -rw-r--r-- 1 root root 5157 1970-01-01 01:00 system_ext_service_contexts -rw-r--r-- 1 root root 7194 1970-01-01 01:00 system_ext_property_contexts -rw-r--r-- 1 root root 3514 1970-01-01 01:00 system_ext_file_contexts drwxr-xr-x 2 root root 3488 1970-01-01 01:00 system_ext drwxr-xr-x 2 root root 3488 1970-01-01 01:00 system_dlkm drwxr-xr-x 2 root root 3488 1970-01-01 01:00 sys drwxr-x--x 2 root root 3488 1970-01-01 01:00 storage -rw-r--r-- 1 root root 1287813 1970-01-01 01:00 sepolicy drwxr-xr-x 2 root root 3488 1970-01-01 01:00 second_stage_resources drwxr-xr-x 2 root root 3488 1970-01-01 01:00 sdcard -rw-r--r-- 1 root root 21189 1970-01-01 01:00 prop.default -rw-r--r-- 1 root root 0 1970-01-01 01:00 product_service_contexts -rw-r--r-- 1 root root 1800 1970-01-01 01:00 product_property_contexts -rw-r--r-- 1 root root 306 1970-01-01 01:00 product_file_contexts drwxr-xr-x 2 root root 3488 1970-01-01 01:00 product drwxr-xr-x 2 root root 3488 1970-01-01 01:00 proc drwxr-xr-x 2 root root 3488 1970-01-01 01:00 postinstall -rw-r--r-- 1 root root 31248 1970-01-01 01:00 plat_service_contexts -rw-r--r-- 1 root root 87362 1970-01-01 01:00 plat_property_contexts -rw-r--r-- 1 root root 43296 1970-01-01 01:00 plat_file_contexts drwxr-xr-x 2 root root 3488 1970-01-01 01:00 oem -rw-r--r-- 1 root root 0 1970-01-01 01:00 odm_property_contexts -rw-r--r-- 1 root root 0 1970-01-01 01:00 odm_file_contexts drwxr-xr-x 2 root root 3488 1970-01-01 01:00 mnt drwxr-xr-x 2 root root 3488 1970-01-01 01:00 metadata drwxr-xr-x 2 root root 3488 1970-01-01 01:00 linkerconfig lrwxrwxrwx 1 root root 16 1970-01-01 01:00 init -> /system/bin/init lrwxrwxrwx 1 root root 11 1970-01-01 01:00 etc -> /system/etc drwxr-xr-x 2 root root 3488 1970-01-01 01:00 dev lrwxrwxrwx 1 root root 12 1970-01-01 01:00 default.prop -> prop.default drwxr-xr-x 2 root root 3488 1970-01-01 01:00 debug_ramdisk drwxr-xr-x 2 root root 3488 1970-01-01 01:00 data_mirror drwxr-x--x 2 root root 3488 1970-01-01 01:00 data lrwxrwxrwx 1 root root 17 1970-01-01 01:00 d -> /sys/kernel/debug dr-xr-xr-x 2 root root 3488 1970-01-01 01:00 config lrwxrwxrwx 1 root root 50 1970-01-01 01:00 bugreports -> /data/user_de/0/com.android.shell/files/bugreports lrwxrwxrwx 1 root root 11 1970-01-01 01:00 bin -> /system/bin drwxr-xr-x 2 root root 3488 1970-01-01 01:00 batinfo drwxr-xr-x 2 root root 3488 1970-01-01 01:00 apex drwxr-xr-x 2 root root 3488 1970-01-01 01:00 acct drwxr-xr-x 3 root root 3488 2024-06-22 10:39 first_stage_ramdisk drwxr-xr-x 3 root root 3488 2024-06-22 10:39 lib drwxr-xr-x 3 root root 3488 2024-06-22 10:39 res drwxr-xr-x 2 root root 3488 2024-06-22 10:39 odm_dlkm drwxr-xr-x 2 root root 3488 2024-06-22 10:39 odm drwxr-xr-x 5 root root 3488 2024-06-22 10:39 system drwxr-xr-x 2 root root 3488 2024-06-22 10:39 vendor_dlkm ASUS_I006D:/data/develop/vendor_boot_a/image/ramdisk #

Now you can change, add, or delete files for the ramdisk in the directory with the uncompressed cpio file.
(see the notes below for how to automatically enable adb in the recovery)

Keep in mind that some links in the ramdisk are absolute symbolic links and other are only mount points for the other partitions.

When everything is done repack the file ramdisk.cpio and create the new image file for the partition vendor_boot:

# only to be sure that the environment variables are set ... # MAGISKBOOT="/data/adb/magisk/magiskboot" CUR_SLOT=$( getprop ro.boot.slot_suffix ) mv ../ramdisk.cpio ../ramdisk.cpio.org # re-create the cpio file with the files for the ramdisk # find . | cpio -o | ${MAGISKBOOT} compress=lz4_legacy - ../ramdisk.cpio # ... and recreate the image for the vendor_boot partition cd .. ${MAGISKBOOT} repack ../vendor_boot${CUR_SLOT}.img

Finally, flash the new image file to the partition vendor_boot:

dd if=./new-boot.img of=/dev/block/by-name/vendor_boot${CUR_SLOT}

and reboot into the recovery mode to check the new ramdisk contents.

Notes

This approach does not work for the StatixOS and the OmniROM. For the OmniROM I recommend to install TWRP into the boot partition.

/data is not automatically mounted if booting into the recovery from /e/, LMODroid, LineageOS 20.x, and LineageOS 21.x.
Workaround: Boot from a TWRP image for the installed OS to also access the files in /data -- see the section How to use TWRP if LineageOS 20.x is installed for how to create a TWRP image for these ROMs

access via adb is not automatically enabled when booting into a recovery for non-debuggable builds from the ROMs mentioned in this post
see https://github.com/LineageOS/android_bootable_recovery for details how to enable the automatic start of the adb after booting into the recovery for these ROMs

Details in short for the insecure version to automatically enable adb

make sure that the value for the variable ro.debuggable in the file /prod.default is 1
and change the value for ro.adb.secure in the file /prop.default for the ramdisk to 0:

ro.adb.secure=0

Additional kernel parameter used while booting into the recovery mode can be added to the line cmdline= in the header file for the vendor_boot partition.

Be aware that the partition vendor_boot is overwritten with each OS update. But the instructions used here can easily be combined in a script in order to apply them automatically after an operating system update, for example.

A script to enable adb in the ramdisk from the partition vendor_boot can be downloaded from here:

enable_adb_in_recovery_ramdisk.sh

To use the script, copy it to the phone and execute it in an adb session. The scripts needs no parameter.

The script enables adb access in the vendor_boot partition for the active slot. The script does not change the partition if adb is already enabled in the ramdisk on the partition.

The script searches the executable magiskboot in the directories /data/adb/magisk and /data/local/tmp; if magiskboot is in another directory set the environment variable MAGISKBOOT before executing the script. To change the ramdisk from another partition set the variable VENDOR_BOOT_PARTITION before executing the script.

Example output of the script

ASUS_I006D:/ # sh /sdcard/Download/enable_adb_in_recovery_ramdisk.sh Initializing the variables ... Enabling adb in the ramdisk on the partition "/dev/block/by-name/vendor_boot_b" ... Checking the pre-requisites ... Using the magiskboot executable "/data/local/tmp/magiskboot" Using the working directory "/data/local/tmp/vendor_boot_b.4546" ... Copying the partition "/dev/block/by-name/vendor_boot_b" to the file "/data/local/tmp/vendor_boot_b.4546/vendor_boot_b.img" ... 196608+0 records in 196608+0 records out 100663296 bytes (96 M) copied, 0.360727 s, 266 M/s Unpacking the image file "/data/local/tmp/vendor_boot_b.4546/vendor_boot_b.img" ... Parsing boot image: [/data/local/tmp/vendor_boot_b.4546/vendor_boot_b.img] VENDOR_BOOT_HDR HEADER_VER [4] RAMDISK_SZ [20625520] DTB_SZ [410902] PAGESIZE [4096] NAME [] CMDLINE [androidboot.console=ttyMSM0 androidboot.hardware=qcom androidboot.memcg=1 androidboot.usbcontroller=a600000.dwc3 cgroup.memory=nokmem,nosocket console=ttyMSM0,115200n8 ip6table_raw.raw_before_defrag=1 iptable_raw.raw_before_defrag=1 loop.max_part=7 lpm_levels.sleep_disabled=1 msm_rtb.filter=0x237 pcie_ports=compat service_locator.enable=1 swiotlb=0 kpti=off buildvariant=userdebug] RAMDISK_FMT [raw] unexpected ASN.1 DER tag: expected SEQUENCE, got APPLICATION [1] (primitive) VBMETA Unpacking the ramdisk from the image file "/data/local/tmp/vendor_boot_b.4546/vendor_boot_b.img" ... The file ramdisk.cpio is compressed The cpio file with the ramdisk contents is compressed with the format "lz4_legacy" Correcting the properties in the file "prop.default" if necessary ... The config entries in the file "prop.default" are: ro.debuggable=1 ro.adb.secure=1 The config entries in the file "prop.default" are now: ro.debuggable=1 ro.adb.secure=0 Recreating the ramdisk.cpio file ... Recreating the file ramdisk.cpio and compressing it using the compression format "lz4_legacy" Repacking the image file for the vendor boot partition in the file "../vendor_boot_b_new.img" ... Parsing boot image: [/data/local/tmp/vendor_boot_b.4546/vendor_boot_b.img] VENDOR_BOOT_HDR HEADER_VER [4] RAMDISK_SZ [20625520] DTB_SZ [410902] PAGESIZE [4096] NAME [] CMDLINE [androidboot.console=ttyMSM0 androidboot.hardware=qcom androidboot.memcg=1 androidboot.usbcontroller=a600000.dwc3 cgroup.memory=nokmem,nosocket console=ttyMSM0,115200n8 ip6table_raw.raw_before_defrag=1 iptable_raw.raw_before_defrag=1 loop.max_part=7 lpm_levels.sleep_disabled=1 msm_rtb.filter=0x237 pcie_ports=compat service_locator.enable=1 swiotlb=0 kpti=off buildvariant=userdebug] RAMDISK_FMT [raw] unexpected ASN.1 DER tag: expected SEQUENCE, got APPLICATION [1] (primitive) VBMETA Repack to boot image: [/data/local/tmp/vendor_boot_b.4546/vendor_boot_b_new.img] HEADER_VER [4] RAMDISK_SZ [20631229] DTB_SZ [410902] PAGESIZE [4096] NAME [] CMDLINE [androidboot.console=ttyMSM0 androidboot.hardware=qcom androidboot.memcg=1 androidboot.usbcontroller=a600000.dwc3 cgroup.memory=nokmem,nosocket console=ttyMSM0,115200n8 ip6table_raw.raw_before_defrag=1 iptable_raw.raw_before_defrag=1 loop.max_part=7 lpm_levels.sleep_disabled=1 msm_rtb.filter=0x237 pcie_ports=compat service_locator.enable=1 swiotlb=0 kpti=off buildvariant=userdebug] 196608+0 records in 196608+0 records out 100663296 bytes (96 M) copied, 4.841879 s, 20 M/s adb successfully enabled in the ramdisk on the partition "/dev/block/by-name/vendor_boot_b" Removing the temporary work directory "/data/local/tmp/vendor_boot_b.4546" ... RC=0 ASUS_I006D:/ #

Trouble Shooting

If the phone hangs booting into recovery after changing the ramdisk on the vendor_boot partition, check the compression method used to recreate the ramdisk - it must be the compression method used to compress the original ramdisk

default.prop in the ramdisk for the recovery is a symbolic link to prop.default but you should not use the link.

If the adb access does not work after booting into the recovery first check if the adb is running at all on the phone using the command

adb devices

Examples:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/Statix/20240106 ] $ adb devices List of devices attached [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/Statix/20240106 ] $
In this case the adb daemon was not started at all - check the rc files in the ramdisk

[ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/e/e-2.0t/test ] $ adb devices List of devices attached M6AIB760D0939LX unauthorized [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/e/e-2.0t/test ]

In this case the adb is running on the phone, but the authorization check for adb is enabled. Double check that the property ro.adb.secure is set to 0 in the file prop.default for the ramdisk.

Test Environment

All tests were done on an ASUS Zenfone 8.

Tests were done with these ROMs:

/e/, OS image file: e-2.0-t-20240514401453-dev-sake.zip

LineageOS 20, OS Image file: lineage-20.0-20240528-nightly-sake-signed.zip

LineageOS 21, OS Image file: sake-2024-05-31-r3-noGMS-ksu-signed-FULL.zip (downloaded from https://github.com/mikooomich/android_device_asus_sake/releases)

LMODroid, OS Image file: LMODroid-4.2-20240613-RELEASE-sake.zip

StatixOS, OS image file: statix_sake-20240508-14-v7.5-UPSIDEDOWNCAKE.zip

OmniROM, OS image file: current latest build for Android 14

History of this entry

23.06.2024

initial release

How to install "fastboot images"

URL: not yet published

How to install "fastboot images"

Some ROMs are provided as "fastboot image" - for example the StatiXOS for ASUS Zenfone 8 is provided as OTA image and as fast boot image; the fastboot images for the StatixOX for the ASUS Zenfone 8 are available here:

https://downloads.statixos.com/14-UPSIDEDOWNCAKE/sake/fastbootimages/

To check if an image file for Android is a fastboot image just check the contents of the ZIP file; the contents of a fastboot image look like this:
[ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/Statix/fastboot_images ] $ unzip -l statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE-img.zip Archive: statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE-img.zip Length Date Time Name --------- ---------- ----- ---- 38 01-01-2008 00:00 android-info.txt 100663296 01-01-2008 00:00 boot.img 25165824 01-01-2008 00:00 dtbo.img 1396736 01-01-2008 00:00 odm.img 2260250624 01-01-2008 00:00 product.img 5184 01-01-2008 00:00 super_empty.img 1430626304 01-01-2008 00:00 system.img 392368128 01-01-2008 00:00 system_ext.img 19451904 01-01-2008 00:00 system_other.img 8192 01-01-2008 00:00 vbmeta.img 4096 01-01-2008 00:00 vbmeta_system.img 956006400 01-01-2008 00:00 vendor.img 100663296 01-01-2008 00:00 vendor_boot.img 31186944 01-01-2008 00:00 vendor_dlkm.img --------- ------- 5317796966 14 files [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/Statix/fastboot_images ] $

Note:

The contents of standard Android OS image for installing via sideload or upgrade looks like this:
[ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/Statix/20240508 ] $ unzip -l statix_sake-20240508-14-v7.5-UPSIDEDOWNCAKE.zip Archive: statix_sake-20240508-14-v7.5-UPSIDEDOWNCAKE.zip signed by SignApk Length Date Time Name --------- ---------- ----- ---- 701 01-01-2009 00:00 META-INF/com/android/metadata 1395 01-01-2009 00:00 META-INF/com/android/metadata.pb 1079 01-01-2009 00:00 apex_info.pb 241 01-01-2009 00:00 care_map.pb 1875651333 01-01-2009 00:00 payload.bin 156 01-01-2009 00:00 payload_properties.txt 1675 01-01-2009 00:00 META-INF/com/android/otacert --------- ------- 1875656580 7 files [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/Statix/20240508 ] $
In these images the image files for partitions are in the file payload.bin. Use a payload dumper like https://github.com/ssut/payload-dumper-go to extract the files in the payload.bin file, e.g.:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/Statix/20240508/statix_sake-20240508-14-v7.5-UPSIDEDOWNCAKE ] $payload-dumper-go -l payload.bin payload.bin: payload.bin Payload Version: 2 Payload Manifest Length: 131336 Payload Manifest Signature Length: 267 Found partitions: boot (101 MB), dtbo (25 MB), odm (1.4 MB), product (2.3 GB), system (883 MB), system_ext (322 MB), vbmeta (8.2 kB), vbmeta_system (4.1 kB), vendor (1.0 GB), vendor_boot (101 MB), vendor_dlkm (38 MB) [ OmniRom 14 Dev - xtrnaw7@t15g /data/backup/ASUS_ZENFONE8/Statix/20240508/statix_sake-20240508-14-v7.5-UPSIDEDOWNCAKE ] $ To extract the files use:
mkdir img_files payload-dumper-go -o $PWD/img_files payload.bin

To install a fastboot image use these commands

Unpack the ZIP file with the fastboot image into an empty directory on your PC and set the environment variable ANDROID_PRODUCT_OUT with the name of that directory:

export ANDROID_PRODUCT_OUT=<directory_with_unpacked_zip_file>

Then boot the phone into the fastbootd using this command on the PC:

adb reboot fastboot

or, if adb is not enabled, execute on the PC while the phone is booted into the bootloader:

fastboot reboot fastboot

and start the installation using the command

fastboot flashall

That's it . The phone will automatically boot into the new OS when the installation is done.

Please note that, depending on the previously installed operating system and the content of the /data partition, it may be necessary to do factory reset the first time you start the newly installed operating system.

To avoid this use the option -w for the fastboot command to wipe the data partition:

fastboot flashall -w

The commands above work in Linux; they should also work in Windows but I can not test that.

Trouble Shooting

The error message

root@t15g:/data/backup/ASUS_ZENFONE8/Statix/fastboot_images/statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE-img# fastboot flashall -------------------------------------------- Bootloader Version...: Post-CS3-1-WW-user Baseband Version.....: Serial Number........: M6AIB760D0939LX -------------------------------------------- Checking 'product' OKAY [ 0.002s] Setting current slot to 'b' FAILED (remote: 'Slot Change is not allowed in Lock State ') fastboot: error: Command failed root@t15g:/data/backup/ASUS_ZENFONE8/Statix/fastboot_images/statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE-img#

is printed if the phone is currently in the bootloader and not in fastboot mode. To fix it reboot the phone into the fastboot mode using the command

fastboot reboot fastboot

and issue the command to flash the phone again (There is no menu entry in the bootloader menu to boot into the fastboot mode but there is a menu entry to boot into the fastboot mode in most recoveries)

On phones like the ASUS Zenfone 8 you might get an error message like this:

root@t15g:~# fastboot flashall -------------------------------------------- Bootloader Version...: unknown Baseband Version.....: Serial Number........: M6AIB760D0939LX -------------------------------------------- Checking 'product' FAILED Device product is 'I006D'. Update requires 'lahaina' or 'ASUS_I006D' or 'sake'. fastboot: error: requirements not met! root@t15g:~#

That's because the product code used for that phone is different in the various ROM images for the phone.

To fix it just add the current product code to the file android-info.txt in the directory with the fast boot images

Example:

# original file contents root@t15g:/data/backup/ASUS_ZENFONE8/Statix/fastboot_images/statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE-img# cat android-info.txt require board=lahaina|ASUS_I006D|sake root@t15g:/data/backup/ASUS_ZENFONE8/Statix/fastboot_images/statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE-img# # changed file contents oot@t15g:/data/backup/ASUS_ZENFONE8/Statix/fastboot_images/statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE-img# cat android-info.txt require board=lahaina|ASUS_I006D|sake|I006D root@t15g:/data/backup/ASUS_ZENFONE8/Statix/fastboot_images/statix_sake-20240106-14-v7.1-UPSIDEDOWNCAKE-img#
see also https://source.android.com/docs/setup/test/running

How to create a TWRP image for LineageOS based ROMs using a script

URL: https://xdaforums.com/t/how-to-use-twrp-if-lineageos-20-x-is-installed.4599721/#post-89582817

How to create a TWRP image for LineageOS based ROMs using a script

The default TWRP image for the ASUS Zenfone 8 will no longer boot if a ROM based on LineageOS 20 or newer is installed on the phone - see How to use TWRP if LineageOS 20.x is installed for details. The post about how to use TWRP for these ROMs includes instructions to create an "ugly" TWRP image for use with these ROMs. But unfortunatley, it doesn't seeem to be possible to create a TWRP image for all LinegageOS based ROMs. Even worse, it looks like every other release of an LineageOS based ROM requires a TWRP image for that version.

I've read some posts and blog entries that TWRP is no longer needed for LineageOS as all necessary tasks can be performed with the recovery from the LineageOS, but I don't think so.

After booting the phone into LineageOS recovery, adb is not enabled by default in the user builds; that may be safe, but it's not very useful because you can't use the adb access to the phone in recovery mode for automatic tasks via script.
I finally figured out how to "fix" this problem (see How to add files to the ramdisk used for the recovery boot), but there is another drawback: /data is not mounted if the phone is booted into the LineageOS recovery and so far I do not know if /data can be mounted while the phone is booted into the LineageOS recovery.

In TWRP, adb is enabled by default and TWRP can mount /data and decrypt the user files if necessasry - this is also true for the "ugly" TWRP images created for the LineageOS based ROMs. And not to mention the other features of TWRP that are not part of the LineageOS recovery (e.g. a local terminal, the various functions of the twrp binary, etc).

Since I need these features to install and customize these ROMs for my development and testing purpose, and I do not want to manually create the TWRP image for each new OS version, I wrote a small Linux shell script to create a TWRP image for these ROMs:

./create_ugly_twrp_image.sh

create_ugly_twrp_image.sh uses the official TWRP image for the phone and the boot partition from the phone (or an image file with the boot partition) to create a TWRP image for that OS.

The usage for the script is:
[ OmniRom 14 Dev - xtrnaw7@t15g / ] $ /data/develop/android/scripts_on_linux/create_ugly_twrp_image.sh -h create_ugly_twrp_image.sh - "create" an TWRP image for an Android OS version not supporting TWRP (e.g an OS based on Lineage 20.x or newer) Usage: create_ugly_twrp_image.sh [-h] [-v] [boot_partition|boot_partition_image_file] [noprop|-p] [ OmniRom 14 Dev - xtrnaw7@t15g / ] $

Verbose usage

[ OmniRom 14 Dev - xtrnaw7@t15g / ] $ /data/develop/android/scripts_on_linux/create_ugly_twrp_image.sh -h -v create_ugly_twrp_image.sh - "create" an TWRP image for an Android OS version not supporting TWRP (e.g an OS based on Lineage 20.x or newer) Usage: create_ugly_twrp_image.sh [-h] [-v] [boot_partition|boot_partition_image_file] [noprop|-p] Parameter: boot_partition - boot partition on the phone to be used for the TWRP image (default: use the boot partition of the active slot) boot_partition_image_file - image file with a boot partition on the PC to be used for the TWRP image noprop - do not copy the properties from the running OS This script must run on PC running the Linux OS; a working TWRP image for the phone is required and access via adb to the phone is required if the script should copy the boot partition for the TWRP image from the phone or change the properties in the ramdisk for the new TWRP image Environment variables used if set: MAGISKBOOT - set this variable to the fully qualified namee of the magiskboot executable if it's not available via the PATH variable (default: search the executable in the PATH) ADB - set this variable to the fully qualified name of the adb executable if it's not available via the PATH variable (default: search the executable in the PATH) ADB_OPTIONS - additional options for adb (there is no default for this variable) TWRP_IMAGE - fully qualified name of the TWRP image to be used to create the new TWRP image WORKDIR - working directory (default: /tmp/create_twrp.$$) NEW_TWRP_FILE - name of the TWRP file to create (default: /tmp/twrp_boot.img) KEEP_WORK_DIR - set this variable to 0 to not delete the temporary directory at script end (default: only keep the temporary directory if an error occured) [ OmniRom 14 Dev - xtrnaw7@t15g / ] $

The script supports some environment variables - see the verbose usage above for details.

To be able to detect this TWRP image later in scripts, the script copies some properties from the current running OS to the new TWRP image; these properties are (as of 26.06.2024):

# properties for the TWRP image - the values for these properties are copied to the TWRP image if they exist in the running OS # PROPERTIES_TO_COPY_TO_THE_TWRP_IMAGE=" ro.lineage.build.version ro.statix.version ro.omni.version ro.lmodroid.build_name ro.build.description ro.build.display.id " The properties to copy are hardcoded in the script; properties that are not defined in the running OS are ignored.

The script uses the executable magiskboot from Magisk to unpack and repack the TWRP images.

The magiskboot executable for x86 in the Magisk apk file is a static linked binary that can be executed on any of the modern Linux distributions for x86 PCs.

To get that executable download the Magisk apk file from https://github.com/topjohnwu/Magisk/releases and extract the executable magiskboot for x86 using the commands:

unzip -p Magisk-v27.0.apk lib/x86/libmagiskboot.so >magiskboot && chmod 755 magiskboot

The script can be downloaded from here: create_ugly_twrp_image.sh

Test Environment

All tests were done on an ASUS Zenfone 8.

Tests were done with these ROMs:

/e/, tested with images files for versions 1.21 and 2.0

LineageOS 20, various OS image files

LineageOS 21, various OS image files (downloaded from https://github.com/mikooomich/android_device_asus_sake/releases)

LMODroid, various OS image files

StatixOS, various OS image files

History of this entry

26.06.2024

initial release

18.07.2024

the link to download the script did not work -- fixed

How to automatically start adbd as root user after starting the phone

URL: https://xdaforums.com/t/how-to-automatically-start-adbd-as-root-user-after-starting-the-phone.4682701/

How to automatically start adbd as root user after starting the phone

In non-debug builds of the Android OS, the adbd always starts as a non-root user, but for some tasks it is necessary to start the adbd as user root.

For example the script ./extract-files.sh from the LineageOS build tools to copy the device dependend files from the phone to the build tree needs direct root access via adb (root access via "su" as implemented by Magisk is not sufficient for that script).

In some Android distributions like for example the LineageOS, is a menu entry Root-Debugging in the developer settings to allow restarting the adbd as user root via the command "adb root" that can be used.
But that option does not survive a reboot of the phone.

To get around this limitation and implement the feature on Android distributions without this developer setting the Magisk module

https://github.com/tiann/adb_root

can be used to start the adbd as root user after rebooting the phone.

That Magisk module also contains an adbd binary so that it also works with Android distributions where root access is disabled in the adbd executable.

Note that there is a bug in the script service.sh from the Magisk module adb_root v1.1. from 17.03.2023:

Fix for service.sh

ASUS_I006D:/data/adb/modules/adb_root # cat /data/adb/modules/adb_root/service.sh #!/system/bin/sh setenforce 0 MODDIR=${0%/*} max_wait=180 interval=1 BOOT_COMPLETED=false while [[ "$max_wait" -gt 0 ]]; do pidof adbd ret=$0 if [ $ret -eq 0 ];then BOOT_COMPLETED=true break fi sleep ${interval} max_wait=$((max_wait-1)) done if [ "$BOOT_COMPLETED" = "false" ];then touch $MODDIR/disable fi

-> "ret=$0" must be replaced with "ret=$?" . Without this change the module always disables itself. This can be fixed after installing the module.

Test Environment

The Magisk module works on most of the Android distributions but not on all.

I tested the Magisk module adb_root with these Android distributions for the ASUS Zenfone 8 (Last update: 22.07.2024):

ROM	Magisk module adb_root works?	Comment
LineageOS 20.x	yes
LineageOS 21.x	yes
ASUS Android 13	yes
OmniROM 13	no	see below for a work around
OmniROM 14	no	see below for a work around
/e/ 2.0x	yes
LMODroid 4.2	yes

Notes

In the LineageOS and LineageOS based OS (like LMODroid or /e/) with installed Magisk this Magisk init script can be used to start the adbd as user root automatically after a reboot:

ASUS_I006D:/data/adb # cat /data/adb/service.d/enable_adb_as_root.sh # # log all output to a log file # exec >"/data/cache/${0##*/}.log" 2>&1 set -x # for using the script in the original Android from ASUS or in the OmniROM uncomment the next line (= disable SELinux) # setenforce 0 magisk resetprop ro.debuggable 1 magisk resetprop service.adb.root 1 stop adbd start adbd ASUS_I006D:/data/adb #To restart the adbd as user root via CLI commands in an adb session execute the script as user root using these commands:

cd /sdcard/Download && nohup /data/adb/service.d/enable_adb_as_root.sh &

Example output:

ASUS_I006D:/ # ps -ef | grep -v grep | grep adbd shell 1201 1 1 14:14:45 ? 00:00:01 adbd --root_seclabel=u:r:su:s0 ASUS_I006D:/ # ASUS_I006D:/ # id uid=0(root) gid=0(root) groups=0(root) context=u:r:magisk:s0 ASUS_I006D:/ # ASUS_I006D:/ # cd /sdcard/Download && nohup /data/adb/service.d/enable_adb_as_root.sh & [3] 4876 # -> the adb session ended because the adbd was restarted # # create a new adb session and check the result: # ASUS_I006D:/ # ASUS_I006D:/ # id uid=0(root) gid=0(root) groups=0(root),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),1078(ext_data_rw),1079(ext_obb_rw),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats),3009(readproc),3011(uhid),3012(readtracefs) context=u:r:su:s0 ASUS_I006D:/ # ASUS_I006D:/ # ps -ef | grep -v grep | grep adbd root 4882 1 0 14:18:23 ? 00:00:00 adbd --root_seclabel=u:r:su:s0 ASUS_I006D:/ #

In the original Android from ASUS and the OmniROM the script only works if SELinux is in the permissive mode.
Alternative only switch SELinux to permissive mode for adb and su. For that use

magiskpolicy --live "permissive { adbd }" magiskpolicy --live "permissive { su }"
instead of "setenforce 0".

To switch back to the adbd running as non-root user execute:

setprop service.adb.root 0; setprop ctl.restart adbd

Use

setprop service.adb.root 1; setprop ctl.restart adbd

to switch back to the adbd running as root user later.

To only disable the authentication for adb connections the Magisk module https://github.com/anasfanani/Adb-Root-Enabler can be used

see also the thread https://xdaforums.com/t/enable-adb-root-from-shell.4298567/ for some further details

History of this entry

21.07.2024

initial release

23.07.2024

added some info about necessary SELinux permissions and how to switch back to the adbd running as non-root user without rebooting

Hints for writing init scripts for Magisk

URL: https://xdaforums.com/t/guide-some-hints-for-writing-init-scripts-for-magisk.4689292/

Hints for writing init scripts for Magisk

Writing and testing Magisk init scripts, either for /data/adb/post-fs-data.d or /data/adb/service.d, can be frustrating as even a small error can result in a non-functioning phone. To fix these errors, the phone must be booted from a recovery image such as TWRP to correct or delete the faulty script. And that all takes time.

A workaround to fix these kind of errors faster while developing the scripts is:

Create a dummy Magisk module for the scripts , e.g.:

mkdir /data/adb/modules/test

Then move the script from /data/adb/post-fs-data.d or /data/adb/service.d to the module directory and create the script post-fs-data.sh or service.sh in the module directory as wrapper for your script:

# # log all output to a log file; the log file using this code is either /data/cache/service.sh.log or /data/cache/post-fs-data.sh.log # exec >"/data/cache/${0##*/}.log" 2>&1 set -x MODDIR=${0%/*} # # disable the module after the next reboot # touch ${MODDIR}/disable # execute your original script # sh -x ${MODDIR}/<your_init_script>
With these commands, the wrapper script creates a log file in the directory /data/cache with STDOUT and STDERR output of your script and automatically deactivates the module on the next reboot. So if something goes wrong with your script, simply restart the phone and fix the problem.

The only downside is that you have to re-enable the module before rebooting the phone (just delete the file â€œdisableâ€ in the module directory)

If your script is working properly, move the script back to /data/adb/post-fs-data.d or /data/adb/service.d and delete the temporary Magisk module.

If the boot of the operating system hangs after installing a new Magisk init script and you did not implement the "boot loop fix" mentioned above check whether the adb access is working (the adbd daemon is started at the beginning of the boot process). If so, open an adb session and edit or delete your script. If that doesn't work, power off the phone and reboot with a recovery image with adb access enabled and /data partition mounted. AFAIK the only recovery that can be used for this purpose is TWRP.

History of this entry

26.04.2024

initial release

29.08.2024

fixed some typos and added some details

Some hints for SELinux usage in Android

URL: not yet published, but the entry is referenced in this post: https://xdaforums.com/t/some-hints-for-selinux-usage-in-android.4683677/

Some hints for SELinux usage in Android

Find below some commands for using SELinux in Android. Due to missing or crippled tools in the Android OS the binaries busybox and magiskpolicy from Magisk are used for some of the commands.

Note that most commands on the phone must be executed as user root.

SELinux contexts for files and directories are stored in the filesystem. Therefore you can not change the SELinux context for files or directories on read-only filesystems.
SELinux cannot be disabled in the Android operating system - it can only be "deactivated" by switching it to permissive mode (in permissive mode, SELinux only issues error messages in the event of access violations, but does not prevent the violation)

For further details see

https://source.android.com/docs/security/features/selinux

https://lineageos.org/engineering/HowTo-SELinux/

A Magisk module with SELinux related executables is available for download -- see the list of Magisk Modules below.

How to get or set the SELinux status

# print the current SELinux status - either Enforcing (= SELinux does not allow access violations) or Permissive (= SELinux only prints messages for access violations) # ASUS_I006D:/ # getenforce Enforcing ASUS_I006D:/ # # set SELinux to Permissive # ASUS_I006D:/ # setenforce 0 ASUS_I006D:/ # # check the result # ASUS_I006D:/ # getenforce Permissive ASUS_I006D:/ # # set SELinux to Enforcing # ASUS_I006D:/ # setenforce 1 ASUS_I006D:/ # # check the result # ASUS_I006D:/ # getenforce Enforcing ASUS_I006D:/ #Or use the command sestatus from the busybox that is part of Magisk:

ASUS_I006D:/ # /data/adb/magisk/busybox sestatus SELinux status:                 enabled SELinuxfs mount:                /sys/fs/selinux Current mode:                   enforcing sestatus: libselinux returns unknown state: No such file or directory 1|ASUS_I006D:/ #

How to view SELinux violations

logcat | grep "avc: denied "

How to create the neccessary SELinux rule to allow a currently not-allowed access

To create the SELinux rules a PC running the Linux OS with the SELinux policy coretuils installed can be used (in Fedora this is the package policycoreutils-python-utils)

All commands must be executed on a PC with a working adb connection to the phone.

In principle, all the information required to create a missing SELInux rule is contained in the SELinux error messages, so you could also create the rule manually. However, it is easier and safer to use the appropriate tools to create the rules.

First create a SELinux policy file with the current active SELinux rules on the phone:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $ adb su - -c shell cat /sys/fs/selinux/policy >android.selinux.policy [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $ # check the created file [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $ file android.selinux.policy android.selinux.policy: SE Linux policy v30 MLS 8 symbols 7 ocons [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $

Then you can create the missing SELinux rules using the Linux command audit2allow:

To create rules for all access denied messages in the current output of logcat use this command:

adb shell logcat -d | grep "avc: denied "   | audit2allow -p android.selinux.policy

Example:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $ adb shell logcat -d | grep "avc: denied "   | audit2allow -p android.selinux.policy #============= priv_app ============== allow priv_app su:unix_stream_socket connectto; #============= su ============== allow su adb_data_file:dir { getattr open read search }; allow su adb_data_file:file getattr; allow su adbd:fd use; allow su shell_exec:file { execute execute_no_trans getattr map open read }; #============= system_suspend ============== allow system_suspend sysfs:dir read; #============= untrusted_app ============== allow untrusted_app su:unix_stream_socket connectto; [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $

To create a specific SELinux rule pipe only the denied message for the issue into audit2allow, example:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $ echo "07-22 19:24:03.077 14888 14888 I logcat : type=1400 audit(0.0:18537): avc: denied { ioctl } for path="/dev/pts/0" dev="devpts" ino=3 ioctlcmd=0x5401 scontext=u:r:su:s0 tcontext=u:object_r:devpts:s0 tclass=chr_file permissive=1" | audit2allow -p android.selinux.policy #============= su ============== allow su devpts:chr_file ioctl; [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $

Finally either add the SELinux rules printed by audit2allow dynamically using magiskpolicy and/or add them to the sepolicy.rule file for an Magisk module to make them persistent.

If no PC with the Linux OS is available try this Perl script:

https://github.com/OpenDarwin-CVS/SEDarwin/blob/master/sedarwin7/src/sedarwin/policycoreutils/audit2allow/audit2allow.perl

Use this URL to download the script using curl or wget

https://raw.githubusercontent.com/OpenDarwin-CVS/SEDarwin/master/sedarwin7/src/sedarwin/policycoreutils/audit2allow/audit2allow.perl

example for using audit2allow.perl:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $ echo "07-22 19:24:03.077 14888 14888 I logcat : type=1400 audit(0.0:18537): avc: denied { ioctl } for path="/dev/pts/0" dev="devpts" ino=3 ioctlcmd=0x5401 scontext=u:r:su:s0 tcontext=u:object_r:devpts:s0 tclass=chr_file permissive=1" | ./audit2allow.perl -i - allow su devpts:chr_file ioctl; [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android ] $

To run the perl script on the phone just install Termux and run it in a Termux session, example:
~ $ logcat -d | grep denied | ./audit2allow.perl allow untrusted_app_27 app_data_file:file relabelfrom; allow untrusted_app_27 shell_test_data_file:dir search; allow untrusted_app_27 untrusted_app_all_devpts:chr_file ioctl; ~ $

How to read or change the SELinux policies in the running OS

Use the binary magiskpolicy from Magisk:
ASUS_I006D:/ # magiskpolicy MagiskPolicy - SELinux Policy Patch Tool Usage: magiskpolicy [--options...] [policy statements...] Options:    --help            show help message for policy statements    --load FILE       load monolithic sepolicy from FILE    --load-split      load from precompiled sepolicy or compile                      split cil policies    --compile-split   compile split cil policies    --save FILE       dump monolithic sepolicy to FILE    --live            immediately load sepolicy into the kernel    --magisk          apply built-in Magisk sepolicy rules    --apply FILE      apply rules from FILE, read and parsed                      line by line as policy statements                      (multiple --apply are allowed)    --print-rules     print all rules in the loaded sepolicy If neither --load, --load-split, nor --compile-split is specified, it will load from current live policies (/sys/fs/selinux/policy) 1|ASUS_I006D:/ #

How to change the SELinux config for a file

Use either the chcon from the Android OS:

ASUS_I006D:/ # chcon --help Toybox 0.8.6-android multicall binary (see toybox --help) usage: chcon [-hRv] CONTEXT FILE... Change the SELinux security context of listed file[s]. -h    Change symlinks instead of what they point to -R    Recurse into subdirectories -v    Verbose ASUS_I006D:/ #

or the chcon from the busybox that is part of Magisk:

ASUS_I006D:/ # /data/adb/magisk/busybox chcon BusyBox v1.36.1-Magisk (2023-09-02 05:30:11 PDT) multi-call binary. Usage: chcon [-chfRv] CONTEXT FILE...     chcon [-chfRv] [-u USER] [-r ROLE] [-l RANGE] [-t TYPE] FILE...     chcon [-chfRv] --reference=RFILE FILE... Change the security context of FILEs to CONTEXT     -v    Verbose     -c    Report changes made     -h    Affect symlinks instead of their targets     -f    Suppress most error messages     --reference RFILE Use RFILE's group instead of using a CONTEXT value     -u USER    Set user/role/type/range in the target security context     -r ROLE     -t TYPE     -l RANGE     -R    Recurse 1|ASUS_I006D:/ #

Note that in init scripts for Magisk or Magisk Modules the chcon from the busybox in Magisk is automatically used.

How to list the current SELinux config

magiskpolicy --print-rules

Example:

ASUS_I006D:/ # magiskpolicy --print-rules | grep permissive permissive adbd permissive su permissive magisk ASUS_I006D:/ #

Alternative use the tool sesearch from the Magisk Module with SELinux tools -- see the list of Magisk Modules below.

Example:

su - -c sesearch --allow /sys/fs/selinux/policy

How to check if a SELinux rule is in place [ Update 27.08.2024]

To check if a SELinux rule is in place use this approach:

# define the SELinux rule#ASUS_I006D:/ # magiskpolicy --live "allow shell magisk unix_dgram_socket sendto"ASUS_I006D:/ # # check if the new SELinux rule is in place#ASUS_I006D:/ # magiskpolicy --print-rules | grep "allow shell magisk unix_dgram_socket { sendto }"allow shell magisk unix_dgram_socket { sendto }ASUS_I006D:/ #

Be aware of neverallow rules

Some SELinux rules can be added to the SELinux policy but they are ignored due to a neverallow rule for that rule example:

ASUS_I006D:/ # magiskpolicy --live "allow untrusted_app property_socket sock_file write"                                                                    ASUS_I006D:/ # ASUS_I006D:/ # magiskpolicy --print-rules | grep "allow untrusted_app property_socket sock_file { write }"                                                 allow untrusted_app property_socket sock_file { write } ASUS_I006D:/ #The rule is is now in place but there is a neverallow rule defined for this rule

# Do not allow untrusted_apps to connect to the property service# or set properties. b/10243159neverallow untrusted_app property_socket:sock_file write;

(see https://android.googlesource.com/platform/external/sepolicy/+/57531cacb40682be4b1189c721fd1e7f25bf3786/untrusted_app.te)

Therefor the rule is ignored by Android

Use the command

su - -c sesearch --neverallow /sys/fs/selinux/policy

to list all active neverallow rules.

The executable sesearch is part of the Magisk Module with SELinux tools - see the list of Magisk Modules below

How to create a file with the current active SELinux policies

To create a file with the current SELinx policies in binary format do

cat /sys/fs/selinux/policy >/sdcard/Download/android.selinux.policy

How to add SELinux policies in the running OSUse magiskpolicy from Magisk to add a new rule to the running SELinux, example:
ASUS_I006D:/ # grep -r system_app /vendor/etc/selinux/ | grep proc_stat /vendor/etc/selinux/vendor_sepolicy.cil:(allow system_app proc_stat (file (ioctl read getattr lock map open watch watch_reads))) ASUS_I006D:/ # # add that policy to the running OS using this command: magiskpolicy "allow system_app proc_stat file { ioctl read getattr lock map open watch watch_reads }"

How to set the SELinux policy to "permissive" for an executable only

magiskpolicy --live "permissive { adbd }"

How to print the SELinux context of the running processes

The parameter -Z of the command ps prints also the SELinux context:

example:
ASUS_I006D:/ # ps -Z LABEL                          USER           PID PPID     VSZ    RSS WCHAN            ADDR S NAME                       u:r:shell:s0                   shell         9738 9632 12890860 3204 __do_sys_+          0 S sh u:r:shell:s0                   shell         9757 9738 12962648 2304 wait_woken          0 S su u:r:magisk:s0                  root          9761   860 12940012 3276 __do_sys_+          0 S - u:r:magisk:s0                  root         23719 9761 12975524 3300 0                   0 R ps ASUS_I006D:/ #

User ps -efZ to list all running processes

How to print the SELinux context of a file

The parameter -Z of the command ls prints also the SELinux context:

ASUS_I006D:/ # ls -lZ /apex/com.android.adbd/bin/adbd                                                                                                                                                  -rwxr-xr-x 1 root shell u:object_r:adbd_exec:s0 2258848 1970-01-01 01:00 /apex/com.android.adbd/bin/adbd ASUS_I006D:/ #

How to change the SELinux context of a file
ASUS_I006D:/ # ls -lZ /data/local/tmp/test.file -rw-rw-rw- 1 root root u:object_r:shell_data_file:s0 0 2024-07-22 17:53 /data/local/tmp/test.file ASUS_I006D:/ # ASUS_I006D:/ # chcon -v u:object_r:adbd_exec:s0 /data/local/tmp/test.file chcon '/data/local/tmp/test.file' to u:object_r:adbd_exec:s0 ASUS_I006D:/ # ASUS_I006D:/ # ls -lZ /data/local/tmp/test.file                                                                                                                                                        -rw-rw-rw- 1 root root u:object_r:adbd_exec:s0 0 2024-07-22 17:53 /data/local/tmp/test.file ASUS_I006D:/ #

How to copy the SELinux context from a file to another file (1)

ASUS_I006D:/ # ls -lZ /data/local/tmp/test.file -rw-rw-rw- 1 root root u:object_r:shell_data_file:s0 0 2024-07-22 18:04 /data/local/tmp/test.file ASUS_I006D:/ # ASUS_I006D:/ # ls -lZ /apex/com.android.adbd/bin/adbd -rwxr-xr-x 1 root shell u:object_r:adbd_exec:s0 2258848 1970-01-01 01:00 /apex/com.android.adbd/bin/adbd ASUS_I006D:/ # ASUS_I006D:/ # chcon -v u:object_r:adbd_exec:s0 /data/local/tmp/test.file chcon '/data/local/tmp/test.file' to u:object_r:adbd_exec:s0 ASUS_I006D:/ # ASUS_I006D:/ # ls -lZ /data/local/tmp/test.file -rw-rw-rw- 1 root root u:object_r:adbd_exec:s0 0 2024-07-22 18:04 /data/local/tmp/test.file ASUS_I006D:/ #

How to copy the SELinux context from a file to another file (2)

ASUS_I006D:/ # ls -lZ /data/local/tmp/test.file -rw-rw-rw- 1 root root u:object_r:shell_data_file:s0 0 2024-07-22 18:00 /data/local/tmp/test.file ASUS_I006D:/ # ASUS_I006D:/ # ls -lZ /apex/com.android.adbd/bin/adbd -rwxr-xr-x 1 root shell u:object_r:adbd_exec:s0 2258848 1970-01-01 01:00 /apex/com.android.adbd/bin/adbd ASUS_I006D:/ # ASUS_I006D:/ # /data/adb/magisk/busybox chcon --reference=/apex/com.android.adbd/bin/adbd /data/local/tmp/test.file ASUS_I006D:/ # ASUS_I006D:/ # ls -lZ /data/local/tmp/test.file -rw-rw-rw- 1 root root u:object_r:adbd_exec:s0 0 2024-07-22 18:00 /data/local/tmp/test.file ASUS_I006D:/ #

How to restore the SELinux context for a file overwritten using a --bind mount

The command restorecon can be used to restore the SELinux context of a file or directory.

|ASUS_I006D:/ # restorecon --help Toybox 0.8.6-android multicall binary (see toybox --help) usage: restorecon [-D] [-F] [-R] [-n] [-v] FILE... Restores the default security contexts for the given files. -D    Apply to /data/data too -F    Force reset -R    Recurse into directories -n    Don't make any changes; useful with -v to see what would change -v    Verbose ASUS_I006D:/ #

For most of the OS files and directories in the Android OS this does not work because the filesystems are mounted read-only. But it can be handy for setting the SELinux context for files or directories overwritten via bind mount - example:

Example

ASUS_I006D:/ # ls -lZ /system/bin/ziptool -rwxr-xr-x 1 root shell u:object_r:system_file:s0 32888 2009-01-01 01:00 /system/bin/ziptool ASUS_I006D:/ # ASUS_I006D:/ # ls -lZ /data/local/tmp/ziptool -rwxr-xr-x 1 root root u:object_r:shell_data_file:s0 32888 2024-07-23 11:47 /data/local/tmp/ziptool ASUS_I006D:/ # ASUS_I006D:/ # mount --bind /data/local/tmp/ziptool /system/bin/ziptool ASUS_I006D:/ # ASUS_I006D:/ # ls -lZ /system/bin/ziptool -rwxr-xr-x 1 root root u:object_r:shell_data_file:s0 32888 2024-07-23 11:47 /system/bin/ziptool ASUS_I006D:/ # # -> restore the SELinux context from the original file using restorecon ASUS_I006D:/ # restorecon /system/bin/ziptool SELinux: Loaded file_contexts ASUS_I006D:/ # ASUS_I006D:/ # ls -lZ /system/bin/ziptool -rwxr-xr-x 1 root root u:object_r:system_file:s0 32888 2024-07-23 11:47 /system/bin/ziptool ASUS_I006D:/ #

How to use sepolicy-inject to add a new SELinux rule to the running OS

Notes:

The commands used here are copied from this web page: https://8ksec.io/android-selinux-internals-part-i-8ksec-blogs/

The command sepolicy-inject is part of the SELinux tools for Android: https://github.com/xmikos/setools-android
The command /system/bin/load_policy is part of the Android OS

First copy the existing SELInux policy to a file
cd /data/local/tmp cp /sys/fs/selinux/policy policy

Add the new rule to that file:

sepolicy-inject -s shell -t rootfs -c file -p read -P policy 'allow shell rootfs:file { read }'

Copy the new file back

cp policy /sys/fs/selinux/policy

and reload the changed set of SELinux rules

/system/bin/load_policy policy

Trouble Shooting

If the permissions and the SELinux context for a file is okay but access is still not working as expected use the commands lsattr and chattr to check / change file attributes managed by these tools
Hopefully no one uses this "tool" on Android but you never know ....

Notes

SELinux explorer (python script for the PC; python 3.8 is required; python 3.12 or newer does not work): https://github.com/Heydarchi/SELinux-Explorer

setools ported to Android: https://github.com/xmikos/setools-android - I created a Magisk module with these binaries - see the list of Magisk modules below
see https://8ksec.io/android-selinux-internals-part-i-8ksec-blogs/ for usage examples for the setools and other information about SELinux

History of this entry

23.07.2024

initial release

27.08.2024

added the instructions to check if a SELinux rule is in place
added infos about neverallow rules
added usage examples for the tool sesearch

How to list bind mounts

URL: https://xdaforums.com/t/how-to-list-bind-mounts.4684276/

How to list bind mounts

bind mounts (mount --bind ...) is a very handy Linux feature to make files on read-only mounted filesystems read-write (see How to make files in /system writable).

bind mounts can also be used to make temporary changes for read-write mounted files without changing the original file, example:

cp --preserve=all /data/data/com.android.dialer/shared_prefs/com.android.dialer.xml /data/local/tmp/com.android.dialer.xml mount --bind /data/local/tmp/com.android.dialer.xml /data/data/com.android.dialer/shared_prefs/com.android.dialer.xml

Now you can change the contents of the file /data/data/com.android.dialer/shared_prefs/com.android.dialer.xml by editing /data/local/tmp/com.android.dialer.xml without overwriting the original file. To "restore" the original file just umount the bind mount:

umount /data/data/com.android.dialer/shared_prefs/com.android.dialer.xml

If something has gone wrong, restart the phone to "restore" the original file.

bind mounts are also used by Magisk to integrate Magisk modules into the OS directories (see How to change files in the directory /system with Magisk and How to change any file or directory using Magisk).

However, these mounts are only partially visible in the output of the mount command, for example:

ASUS_I006D:/data/local/tmp # cat $PWD/a A ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a1 A1 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # mount --bind $PWD/a1 $PWD/a ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a A1 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # mount | grep $PWD/a /dev/block/dm-37 on /data/local/tmp/a type f2fs (rw,lazytime,seclabel,nosuid,nodev,noatime,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,extent_cache,mode=adaptive,active_logs=6,reserve_root=32768,resuid=0,resgid=1065,inlinecrypt,alloc_mode=default,fsync_mode=nobarrier) ASUS_I006D:/data/local/tmp #

The bind mount is visible in the output of the mount command, but there is no information about the source for the bind mount in the output of the mount command.

To get the source for the bind mount, the contents of /proc/self/mountinfo can be used, example:

ASUS_I006D:/data/local/tmp # grep $PWD/a /proc/self/mountinfo 62606 126 253:37/local/tmp/a1/data/local/tmp/a rw,nosuid,nodev,noatime shared:42 - f2fs /dev/block/dm-37 rw,lazytime,seclabel,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,extent_cache,mode=adaptive,active_logs=6,reserve_root=32768,resuid=0,resgid=1065,inlinecrypt,alloc_mode=default,fsync_mode=nobarrier ASUS_I006D:/data/local/tmp #

The source for the bind mount in the contents from /proc/self/mountinfo is in the 4th field of the output: This is the absolute path in the mounted filesystem (/local/tmp/a1)with the major and minor device numbers printed in the third field (253:37).

These commands can be used to obtain the file system with this combination of major and minor device number:

ASUS_I006D:/ # ls -ld /sys/dev/block/253:37 lrwxrwxrwx 1 root root 0 1970-01-13 02:37 /sys/dev/block/253:37 -> ../../devices/virtual/block/dm-37 ASUS_I006D:/ # ASUS_I006D:/ # df -h | grep /dm-37 /dev/block/dm-37 108G 6.9G 101G 7% /data ASUS_I006D:/ #

Or, in one command:

ASUS_I006D:/ # ( export CUR_MOUNT="/data/local/tmp/a" ; df -h | grep "/dev/block/$( ls -ld /sys/dev/block/$( grep ${CUR_MOUNT} /proc/self/mountinfo | awk '{ print $3 }' ) | awk -F/ '{ print $NF }' )" | awk '{ print $NF }' ) /data ASUS_I006D:/ #

So, /data/local/tmp/a is currently bind mounted to /data/local/tmp/a1.

The commands can also be used to check which file is being used if a file is part of several Magisk modules, for example:

# the file "/system/bin/Test11" is part of three Magisk Modules: ASUS_I006D:/ # ls -l /data/adb/modules/*/system/bin/Test11 -rw-r--r-- 1 root root 5 2024-07-27 12:39 /data/adb/modules/DebugTools/system/bin/Test11 -rw-r--r-- 1 root root 5 2024-07-27 12:39 /data/adb/modules/bootctl-binary/system/bin/Test11 -rw-r--r-- 1 root root 5 2024-07-27 12:39 /data/adb/modules/vim/system/bin/Test11 ASUS_I006D:/ # ASUS_I006D:/ # ls -l /system/bin/Test11 -rw-r--r-- 1 root root 5 2024-07-27 12:39 /system/bin/Test11 ASUS_I006D:/ # ASUS_I006D:/ # grep /system/bin/Test11 /proc/self/mountinfo 891 890 253:37 /adb/modules/bootctl-binary/system/bin/Test11 /system/bin/Test11 ro,noatime shared:15 - f2fs /dev/block/dm-37 rw,lazytime,seclabel,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,extent_cache,mode=adaptive,active_logs=6,reserve_root=32768,resuid=0,resgid=1065,inlinecrypt,alloc_mode=default,fsync_mode=nobarrier ASUS_I006D:/ #

The number of "bind mounts" for a file or directory is not limited but only the last bind mount is active. But even in this case the approach listed above can be used.

Details

ASUS_I006D:/data/local/tmp # cat $PWD/a1 A1 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a2 A2 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # mount --bind $PWD/a1 $PWD/a ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a A1 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a1 A1 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a2 A2 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # mount --bind $PWD/a2 $PWD/a ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a A2 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a1 A2 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a2 A2 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # grep $PWD/a /proc/self/mountinfo | sed "s/$/\n/g" 62606 126 253:37 /local/tmp/a1 /data/local/tmp/a rw,nosuid,nodev,noatime shared:42 - f2fs /dev/block/dm-37 rw,lazytime,seclabel,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,extent_cache,mode=adaptive,active_logs=6,reserve_root=32768,resuid=0,resgid=1065,inlinecrypt,alloc_mode=default,fsync_mode=nobarrier 68352 62606 253:37 /local/tmp/a2 /data/local/tmp/a rw,nosuid,nodev,noatime shared:42 - f2fs /dev/block/dm-37 rw,lazytime,seclabel,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,extent_cache,mode=adaptive,active_logs=6,reserve_root=32768,resuid=0,resgid=1065,inlinecrypt,alloc_mode=default,fsync_mode=nobarrier 68357 126 253:37 /local/tmp/a2 /data/local/tmp/a1 rw,nosuid,nodev,noatime shared:42 - f2fs /dev/block/dm-37 rw,lazytime,seclabel,background_gc=on,discard,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,extent_cache,mode=adaptive,active_logs=6,reserve_root=32768,resuid=0,resgid=1065,inlinecrypt,alloc_mode=default,fsync_mode=nobarrier ASUS_I006D:/data/local/tmp #

The mounts listed here are in the order they were executed so the active source for the bind mount for /data/local/tmp/a is /data/local/tmp/a2.

Note that there is also a bind mount in place now for the other source for the bind mount, /data/local/tmp/a1:

ASUS_I006D:/data/local/tmp # cat $PWD/a A2 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a1 A2 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a2 A2 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # echo "Test" > $PWD/a ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a Test ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a1 Test ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a2 Test ASUS_I006D:/data/local/tmp # # remove the bind mounts and check the contents of the files ASUS_I006D:/data/local/tmp # umount $PWD/a ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # umount $PWD/a ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a a ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a1 A1 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat $PWD/a2 Test ASUS_I006D:/data/local/tmp #

In Linux there is the command findmnt that can be used to list also the bind mounts, example:
[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts ] $ findmnt | egrep "\[" â”œâ”€/var/tmp/phonedata.tar                                 /dev/mapper/luks-0c22d523-35b4-4afb-b00a-43dbb161ed84[/var/tmp/x1.tar]                  ext4            rw,relatime,seclabel â””â”€/data/develop/android/scripts/a                        /dev/mapper/luks-0c22d523-35b4-4afb-b00a-43dbb161ed84[/data/develop/android/scripts/a1] ext4            rw,relatime,seclabel [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts ] $

or

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts ] $ findmnt -o TARGET,FSROOT | egrep -v "/$" TARGET                                                   FSROOT â”œâ”€/var/tmp/phonedata.tar                                 /var/tmp/x1.tar â””â”€/data/develop/android/scripts/a                        /data/develop/android/scripts/a1 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts ] $

findmnt does not exist in the Android OS and until now I did not manage to compile the source code of findmnt for the Android OS.

Therfore I wrote a little shell script to list all bind mounts:

list_bind_mounts.sh

The parameter for the script are the files to check. Without a parameter the script lists all active bind mounts.

Examples
ASUS_I006D:/ # /data/local/tmp/list_bind_mounts.sh /data/local/tmp/a ; echo "RC=$?" /data/local/tmp/a    -> /data/local/tmp/a1 RC=0 ASUS_I006D:/ # ASUS_I006D:/ # /data/local/tmp/list_bind_mounts.sh /data/local/tmp/a1 ; echo "RC=$?" /data/local/tmp/a1 is not bind mounted RC=1 ASUS_I006D:/ # ASUS_I006D:/ # /data/local/tmp/list_bind_mounts.sh /data/local/tmp/a1 /data/local/tmp/a   ; echo "RC=$?" /data/local/tmp/a1 is not bind mounted /data/local/tmp/a    -> /data/local/tmp/a1 RC=1 ASUS_I006D:/ # ASUS_I006D:/data/local/tmp # ./list_bind_mounts.sh # mounts to check found # Checking 20 mount entries ... /data_mirror/data_ce/null -> /data/user          /data_mirror/data_de/null -> /data/user_de       /data_mirror/misc_ce/null -> /data/misc_ce       /data_mirror/misc_de/null -> /data/misc_de       /data_mirror/cur_profiles -> /data/misc/profiles/cur /data_mirror/ref_profiles -> /data/misc/profiles/ref /data_mirror/data_ce/null/0 -> /data/data          /data/user/0         -> /data/data          /mnt/pass_through/0/emulated -> /data/media         /mnt/androidwritable/0/emulated/0/Android/data -> /data/media/0/Android/data /mnt/installer/0/emulated/0/Android/data -> /data/media/0/Android/data /mnt/user/0/emulated/0/Android/data -> /data/media/0/Android/data /storage/emulated/0/Android/data -> /data/media/0/Android/data /mnt/androidwritable/0/emulated/0/Android/obb -> /data/media/0/Android/obb /mnt/installer/0/emulated/0/Android/obb -> /data/media/0/Android/obb /mnt/user/0/emulated/0/Android/obb -> /data/media/0/Android/obb /storage/emulated/0/Android/obb -> /data/media/0/Android/obb /data/local/tmp/a    -> /data/local/tmp/a1 /data/local/tmp/a    -> /data/local/tmp/a2 /data/local/tmp/a1   -> /data/local/tmp/a2 # 20 bind mount(s) found ASUS_I006D:/data/local/tmp #

All info messages of the script are written to STDERR - to suppress them use 2>/dev/null , example

/data/local/tmp/list_bind_mounts.sh /system/bin/* 2>/dev/null

ASUS_I006D:/ # /data/local/tmp/list_bind_mounts.sh /system/bin/* 2>/dev/null /system/bin/Test11 -> /data/adb/modules/bootctl-binary/system/bin/Test11 /system/bin/aapt -> /data/adb/modules/androidsdk34_0_3/system/bin/aapt /system/bin/aapt2 -> /data/adb/modules/androidsdk34_0_3/system/bin/aapt2 /system/bin/adb -> /data/adb/modules/androidsdk34_0_3/system/bin/adb /system/bin/aidl -> /data/adb/modules/androidsdk34_0_3/system/bin/aidl /system/bin/bootctl -> /data/adb/modules/bootctl-binary/system/bin/bootctl /system/bin/bootctl.bin -> /data/adb/modules/bootctl-binary/system/bin/bootctl.bin /system/bin/btop -> /data/adb/modules/btop/system/bin/btop /system/bin/btop.bin -> /data/adb/modules/btop/system/bin/btop.bin /system/bin/dexdump -> /data/adb/modules/androidsdk34_0_3/system/bin/dexdump /system/bin/dmtracedump -> /data/adb/modules/androidsdk34_0_3/system/bin/dmtracedump /system/bin/e2fsdroid -> /data/adb/modules/androidsdk34_0_3/system/bin/e2fsdroid /system/bin/etc1tool -> /data/adb/modules/androidsdk34_0_3/system/bin/etc1tool /system/bin/fastboot -> /data/adb/modules/androidsdk34_0_3/system/bin/fastboot /system/bin/fdisk -> /data/adb/modules/DebugTools/system/bin/fdisk /system/bin/gdisk -> /data/adb/modules/DebugTools/system/bin/gdisk /system/bin/hprof-conv -> /data/adb/modules/androidsdk34_0_3/system/bin/hprof-conv /system/bin/make_f2fs -> /data/adb/modules/androidsdk34_0_3/system/bin/make_f2fs /system/bin/make_f2fs_casefold -> /data/adb/modules/androidsdk34_0_3/system/bin/make_f2fs_casefold /system/bin/mke2fs -> /data/adb/modules/androidsdk34_0_3/system/bin/mke2fs /system/bin/nano -> /data/adb/modules/nano-ndk/system/bin/nano /system/bin/nano.bin -> /data/adb/modules/nano-ndk/system/bin/nano.bin /system/bin/ncat -> /data/adb/modules/DebugTools/system/bin/ncat /system/bin/ngrep -> /data/adb/modules/DebugTools/system/bin/ngrep /system/bin/sload_f2fs -> /data/adb/modules/androidsdk34_0_3/system/bin/sload_f2fs /system/bin/split-select -> /data/adb/modules/androidsdk34_0_3/system/bin/split-select /system/bin/sqlite3 -> /data/adb/modules/androidsdk34_0_3/system/bin/sqlite3 /system/bin/strace -> /data/adb/modules/DebugTools/system/bin/strace /system/bin/sudo -> /data/adb/modules/DebugTools/system/bin/sudo /system/bin/tcpdump -> /data/adb/modules/DebugTools/system/bin/tcpdump /system/bin/vim -> /data/adb/modules/vim/system/bin/vim /system/bin/vim.bin -> /data/adb/modules/vim/system/bin/vim.bin /system/bin/vimtutor -> /data/adb/modules/vim/system/bin/vimtutor /system/bin/xxd -> /data/adb/modules/vim/system/bin/xxd /system/bin/zipalign -> /data/adb/modules/androidsdk34_0_3/system/bin/zipalign 1|ASUS_I006D:/

Set the environment variable VERBOSE to 0 to get some debug messages from the script:

VERBOSE=0 /data/local/tmp/list_bind_mounts.sh /system/bin/btop /system/bin/hid

ASUS_I006D:/ # VERBOSE=0 /data/local/tmp/list_bind_mounts.sh /system/bin/btop /system/bin/hid # Checking 2 mount entries ... Processing "/system/bin/btop" ... + echo '253:37 /adb/modules/btop/system/bin/btop /system/bin/btop ####' + cut -f1 -d ' ' + MAJOR_MINOR=253:37 + echo '253:37 /adb/modules/btop/system/bin/btop /system/bin/btop ####' + cut -f2 -d ' ' + MOUNT_SOURCE=/adb/modules/btop/system/bin/btop + echo '253:37 /adb/modules/btop/system/bin/btop /system/bin/btop ####' + cut -f3 -d ' ' + MOUNT_TARGET=/system/bin/btop + set +x MOUNT_SRC_ROOT_DEV is /data /system/bin/btop -> /data/adb/modules/btop/system/bin/btop Processing "/system/bin/hid" ... /system/bin/hid is not bind mounted 1|ASUS_I006D:/ #

The script can be downloaded from here: list_bind_mounts.sh.

Notes

see also

https://unix.stackexchange.com/questions/18048/list-only-bind-mounts
https://unix.stackexchange.com/questions/295525/how-is-findmnt-able-to-list-bind-mounts

History of this entry

27.07.2024

initial release

28.07.2024

added the documentation for the script list_bind_mounts.sh

How to apply an Android patchlevel to the repositories for the StatiXOS

URL: https://xdaforums.com/t/how-to-apply-an-android-security-patchlevel-to-a-local-repository-for-the-statixos.4686280

How to apply an Android patchlevel to the repositories for the StatiXOS

To apply an Android patchlevel to the repositories for the StatiXOS these instructions can be used:

First get the tag for the patch from this page:

https://source.android.com/docs/setup/reference/build-numbers#source-code-tags-and-builds

Then issue these commands in the top level of the repositories for the StatiXOS:

# the branch for the new Android patchlevel # NEW_BRANCH="android-14.0.0_r55" # correct the branch in the file .repo/manifests/default.xml # sed -i -e "s/android-14.0.0_r[0-9][0-9]/android-${NEW_BRANCH}/g" .repo/manifests/default.xml # Apply the patches to the AOSP repositories # time python vendor/statix/scripts/merge-aosp.py ${NEW_BRANCH} # fix all merge conflicts printed by the command above # Update the repo build/release # cd build/release && git pull aosp android-${NEW_BRANCH}

Now create the new OS image, e.g. to create the OS image for the ASUS Zenfone 8 use:

. build/envsetup.sh time brunch statix_sake-apa2a-user

A small script to apply an Android patchlevel to the repositories for the StatixOS is available here: apply_security_level_to_StatixOS_repos.sh

Notes

see also here How to apply the Android Security patches to a local repository

The XDA thread for the StatiXOS, https://xdaforums.com/t/closed-rom-upsidedowncake-sake-14-statixos-v7-10.4500497/, is closed since 08.07.2024.

History of this entry

11.08.2024

initial release

12.08.2024

add the link to the script to apply an Android patchevel

How to start or stop an app via CLI command

URL: not yet published

How to start or stop an app via CLI command

To start or stop an app in a adb or terminal session the command am can be used.

To stop an app via CLI command in a terminal or adb session on the phone use

am force-stop <app>

e.g. to stop the Jecelyin editor use:

am force-stop com.jecelyin.editor

To start an app via the CLI command am, the app action for starting the app is required.

A simple method to get that action is:

Start the app via GUI on the phone and bring it in the foreground.

Then execute this command in an terminal session:

dumpsys window displays | grep -i mCurrentFocus

e.g.

ASUS_I006D:/storage/emulated/0/Download # dumpsys window displays | grep -i mCurrentFocus                                                                                                                                                                           mCurrentFocus=null mCurrentFocus=Window{f9018d2 u0 com.jecelyin.editor/com.jecelyin.editor.JecEditor}ASUS_I006D:/storage/emulated/0/Download #

-> the parameter for the am command to start the app is com.jecelyin.editor/com.jecelyin.editor.JecEditor.

e.g.

am start -n com.jecelyin.editor/com.jecelyin.editor.JecEditor

Another method to get the action used to start the app is :

cmd package resolve-activity --brief <package_name>

e.g.
ASUS_I006D:/storage/emulated/0/Download # cmd package resolve-activity --brief com.jecelyin.editor | tail -1com.jecelyin.editor/.JecEditorASUS_I006D:/storage/emulated/0/Download #

To start an app without knowing the action for starting the app the tool monkey can be used to start the app:

monkey -p <app_name> -c android.intent.category.LAUNCHER 1

e.g.

monkey -p com.jecelyin.editor -c android.intent.category.LAUNCHER 1

Notes

I do not know of an "official" command to list all actions defined by an app.

To list all actions defined by an app this command can be used:

dumpsys package <package_name> | sed -n "/[ \t]*Actions:/,/^$/p"

e.g.

dumpsys package com.jecelyin.editor | sed -n "/[ \t]*Actions:/,/^$/p"

ASUS_I006D:/storage/emulated/0/Download # dumpsys package com.jecelyin.editor | sed -n "/[ \t]*Actions:/,/^$/p" Non-Data Actions: android.intent.action.MAIN: 7fe078f com.jecelyin.editor/.JecEditor filter 510fa1c Action: "android.intent.action.MAIN" Category: "android.intent.category.LAUNCHER" com.googlecode.android_scripting.action.EDIT_SCRIPT: 7fe078f com.jecelyin.editor/.JecEditor filter d419c08 Action: "com.googlecode.android_scripting.action.EDIT_SCRIPT" Category: "android.intent.category.DEFAULT" android.intent.action.SEARCH: 7fe078f com.jecelyin.editor/.JecEditor filter 35d8dab Action: "android.intent.action.SEARCH" Category: "android.intent.category.DEFAULT" MIME Typed Actions: android.intent.action.SEND: 7fe078f com.jecelyin.editor/.JecEditor filter bff83fa Action: "android.intent.action.SEND" Category: "android.intent.category.DEFAULT" StaticType: "text" mPriority=0, mOrder=0, mHasStaticPartialTypes=true, mHasDynamicPartialTypes=false android.intent.action.VIEW: 7fe078f com.jecelyin.editor/.JecEditor filter 2173625 Action: "android.intent.action.VIEW" Category: "android.intent.category.DEFAULT" Category: "android.intent.category.BROWSABLE" Scheme: "file" Scheme: "content" StaticType: "application/x-*" StaticType: "application/xml" StaticType: "application/postscript" StaticType: "application/plain" StaticType: "application/x-tcl" StaticType: "application/x-javascript" StaticType: "application/inf" StaticType: "application/octet-stream" StaticType: "text" mPriority=0, mOrder=0, mHasStaticPartialTypes=true, mHasDynamicPartialTypes=falseASUS_I006D:/storage/emulated/0/Download #

I found a well done oneliner to list all activities of an app on the webpage https://sps-support.honeywell.com/s/article/How-to-get-the-package-and-main-class-name-of-an-application

dumpsys package | grep -Eo "^[[:space:]]+[0-9a-f]+[[:space:]]+<app_name>/[^[:space:]]+" | grep -oE "[^[:space:]]+$"

Examples:

ASUS_I006D:/storage/emulated/0/Download # dumpsys package | grep -Eo "^[[:space:]]+[0-9a-f]+[[:space:]]+com.jecelyin.editor/[^[:space:]]+" | grep -oE "[^[:space:]]+$" | sort | uniqcom.jecelyin.editor/.JecEditorASUS_I006D:/storage/emulated/0/Download # ASUS_I006D:/storage/emulated/0/Download # dumpsys package | grep -Eo "^[[:space:]]+[0-9a-f]+[[:space:]]+com.termoneplus/[^[:space:]]+" | grep -oE "[^[:space:]]+$" | sort | uniqcom.termoneplus/.RemoteSessioncom.termoneplus/.TermActivitycom.termoneplus/.TermHerecom.termoneplus/.shortcuts.AddShortcutcom.termoneplus/.shortcuts.FileSelectioncom.termoneplus/androidx.profileinstaller.ProfileInstallReceivercom.termoneplus/jackpal.androidterm.RemoteInterfacecom.termoneplus/jackpal.androidterm.RunScriptcom.termoneplus/jackpal.androidterm.RunShortcutcom.termoneplus/jackpal.androidterm.TermServiceASUS_I006D:/storage/emulated/0/Download #

History of this entry

17.08.2024

initial release

18.08.2024

added link to the documentation of the monkey command

How to use a log host within the Android operating system

URL: not yet published

Note: As of 28.08.2024 this section is still work in progress

How to use a log host within the Android operating system

AFAIK Android does not support using a loghost for the log messages.

Therefor I compiled the syslogd and the logger executables from the GNU inetutils (https://www.gnu.org/software/inetutils/inetutils.html) for Android running on a ARM64 CPU and created a Magisk Module to install them.

The Magisk Module can be downloaded from here: syslogd_2.5.0.zip.

The Magisk Module installs these files:

File / Directory	Content	Comment
/system/bin/server/syslogd	the syslogd binary
/system/bin/nlogger	the logger binary	this is the executable logger from the GNU inetuils (There is already a binary called logger in the Android OS )
/system/bin/restart_syslogd	a script to restart the syslogd	a symbolic link to ./start_syslogd
/system/bin/start_syslogd	a script to start the syslogd
/system/bin/stop_syslogd	a script to stop the syslogd	a symbolic link to ./start_syslogd

/system/etc/syslog.conf_with_local_files	an example syslog config file to log a loghost and to local files
/system/etc/syslog.conf	a syslog config file to log only to the loghost	this is the default syslog config file
/system/etc/syslog.d	the directory for additional syslog config files	additional syslog config files must use the suffic .conf

/system/etc/hosts	a writable host file to be able to define the loghost	The file /system/etc/hosts is not changed if already replaced by another Magisk Module.

The Magisk Module also installs the Magisk init script /data/adb/modules/syslogd/service.sh to start the syslogd after each reboot. To disable the automatic start of the syslogd by this script create the file /data/local/tmp/disable_syslogd.
The script /data/adb/modules/syslogd/service.sh logs to the file /data/local/tmp/start_syslog_init_script.log.

To manually start the syslogd the script /system/bin/start_syslogd can be used.

The usage for the script is:

start_syslogd -h

ASUS_I006D:/ # /system/bin/start_syslogd -h Usage: /system/bin/start_syslogd [additional_parameter_for_the_syslogd] The script uses these environment variables: SYSLOGD_BINARY - name of the syslogd to use; the current value is /system/bin/server/syslogd SYSLOGD_CONF_FILE -name of the config file for the syslod; the current value is /system/etc/syslog.conf SYSLOGD_CONF_DIR - name of the directory with additional syslogd config files; the current value is /system/etc/syslog.d SYSLOGD_PIDFILE - name of the PID file to use; the current value is /data/local/tmp/syslogd.pid LOGFILE - name of the log file to use; the current value is /datalocal/tmp/start_syslogd.log Use the parameter "--help" to print the usage help of the syslogd 1|ASUS_I006D:/ #

The syslogd must be started by the root user; therefore the script uses “su - -c <command>” to start the syslogd if it is started by a non-root user.

Since the Android Linux kernel is very sluggish, only the forwarding of all messages to the loghost is configured in the syslog config file /system/etc/syslog.conf in the Magisk module. The only messages that are written to the phone's local files are the messages for the syslog facility auth.info . The file /system/etc/syslog.conf should therefore be changed for the desired configuration.

The file /system/etc/syslog.conf_with_local_files is an example syslog config file to log to local files and the loghost

Be aware that the syslogd must be restarted to re-read the config files.

The script

/system/bin/restart_syslogd

can be used to restart the syslogd.

The script

/system/bin/stop_syslogd

can be used to stop the syslogd.

To test, if the syslog works this command can be used:

nlogger -p auth.info "message"

Example:

130|ASUS_I006D:/ # date ; nlogger -p auth.info "Hallo ihr da draussen" Mon Aug 26 15:29:00 CEST 2024 ASUS_I006D:/ # ASUS_I006D:/ # grep Hallo /data/local/tmp/log/messages | tail -1 Aug 26 15:29:00 localhost root: Hallo ihr da draussen ASUS_I006D:/ #

To write the messages to the loghost, the hostname "loghost" must be defined in the file /system/etc/hosts, e.g.:

ASUS_I006D:/ # grep loghost /system/etc/hosts192.168.1.126     loghostASUS_I006D:/ # Or use a hostname defined by your DNS server instead of loghost in the syslog config file.

To forward all logcat error messages to the syslogd on the phone a command like this could be used:

# clear the old messages from the buffer and forward all new error messages via syslogd to the loghost # logcat -c ; logcat " *:E " | while read line; do /system/bin/nlogger -p local1.info -- "$line" ; done

or to forward all logcat messages use:

# clear the old messages from the buffer and forward all new messages to the syslogd # logcat -c ; logcat | while read line; do /system/bin/nlogger -p local1.info -- "$line" ; done

":E" in the first example is the priority of the message; the known priorities in Android are (ordered from lowest to highest priority):

    V: Verbose (lowest priority)
    D: Debug
    I: Info
    W: Warning
    E: Error
    F: Fatal
    S: Silent (highest priority, on which nothing is ever printed)

Note

To forward all logcat messages to another loghost, this logger command can be used (the local syslogd on the phone is not necessary for this command):

# (optional) clear the old messages from the buffer and forward all new messages to the loghost 192.168.1.126 # logcat -c ; logcat | while read line; do /system/bin/nlogger -h 192.168.1.126 -p local1.info -- "$line" ; done

Use a syslog facility that is not used for standard log messages on the loghost, such as local1.info, so that the loghost can write the messages to a unique file.

Trouble Shooting

It seems that Magisk does the bind mounts for files in the directory /system after executing the service scripts of an Magisk Module. The syslogd reads the file /system/etc/hosts to determine the name of the loghost to use. Therefore, the service script for starting the syslogd waits some seconds before it starts the syslogd:

   echo "Waiting now ${WAIT_TIME} seconds before starting the syslogd to make sure the bind mount for /system/etc/hosts is in place ..."   ( sleep ${WAIT_TIME} ; ${SYSLOGD_START_SCRIPT} ) &

If the syslogd starts after a reboot of the phone but does not forward messages to the loghost, most probably it was started too quickly. In this case, increase the number of seconds to wait until the syslogd has started.
This is the variable WAIT_TIME in the script /data/adb/modules/syslogd/service.sh:

ASUS_I006D:/ # grep WAIT_TIME= /data/adb/modules/syslogd/service.sh                                                                                                                                                         WAIT_TIME=20ASUS_I006D:/ #

To check whether a too fast start of the log host is the problem, restart the syslogd with the script /system/bin/restart_syslogd in an adb session after the operating system has started on the phone.

Another check is to write a log message to a loghost using the binary nlogger on the phone - e.g. to check the connection to the loghost with the IP 192.168.1.126 do:

nlogger -h 192.168.1.126 user.info "Ist da jemand?"

and check the logfiles on the loghost.

Additional SELinux policies are necessary for each program not running as root user, that wants to write syslog messages. The necessary SELinux policies for the nlooger command executed in an adb session are already added in the syslog start script:

      echo "Add the SELinux policy for using the syslogd by the process shell ..."      ${PREFIX} magiskpolicy --live "allow shell device sock_file write"      ${PREFIX} magiskpolicy --live "allow shell magisk unix_dgram_socket sendto"

If SELinux authorizations are missing, an error message like this is displayed:

~ $ logger -p auth.info “hallo you out there”logger: cannot establish a connection: Permission denied~ $

To make sure that the problem is the missing SELinux policies, temporarily disable SELinux with

setenforce 0

and try the logger command again. Or run the command as user root.

~~Note that I have not yet succeeded in assigning the required SELinux authorizations to a third party~~ ~~app.~~
(see https://xdaforums.com/t/problem-selinux-rule-added-via-magiskpolicy-seems-not-to-work.4688952/ for details)

Update 29.8.2024

It's not allowed in Android to grant the necessary SELinux perrmissions to write syslog messages to third-party apps (= untrusted_apps)

An ugly workaround for other shells running on the phone is:

restart the syslogd with the parameter --inet .

/system/bin/restart_syslogd --inet

Use the host parameter of the logger command in the other shell, example command in a Termux Session:

logger -h localhost -p auth.info "Ich auch"

or, for shells without a logger command:

/system/bin/nlogger -h localhost -p auth.info "So geht's auch"

History of this entry

28.08.2024

initial release

Some hints for creating Magisk Modules

URL: not yet published

Some hints for creating Magisk Modules

Here are some tips for creating Magisk modules that I have found very useful when developing Magisk modules and that might be useful for others as well.

This is NOT a general HowTo to create Magisk Modules. See https://topjohnwu.github.io/Magisk/guides.html for the "official" instructions on how to create a Magisk Module.

Creating a Magisk Module

The easiest way to create a Magisk module is to copy an existing module. Simply unpack the ZIP file for an existing module and delete all files and directories in the directory in which you unpacked the ZIP file, with the exception of these files:

./META-INF
./META-INF/com
./META-INF/com/google
./META-INF/com/google/android
./META-INF/com/google/android/update-binary
./META-INF/com/google/android/updater-script
./uninstall.sh
./module.prop
./customize.sh
./common
./common/functions.sh

The file module.prop contains the description for the Module and must be changed; the script customize.sh is executed by Magisk while installing the module (if it exists)

Do not change any of the other files for the Magisk Module.

Example module.prop file

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/ModuleSrc/syslogd ] $ cat module.propid=syslogdname=syslogdversion=2.5.0versionCode=3author=<your_email_address>description=syslog from the GNU inetuils[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/ModuleSrc/syslogd ] $

Scripts for the Magisk Module

There are three scripts in a Magisk Module that are used by Magisk:

customize.sh
service.sh
post-fs-data.sh

All scripts must be in the top directory of the Magik Module. All scripts are optional.

The script customize.sh is only executed once while the Magisk Module is being installed; to create a script that is executed each time while the Magisk Module is loaded, create a script called service.sh in the root directory of the Magisk Module (or, if necessary, post-fs-data.sh - but in tis case read the Magisk documentation carefully before using this script). All scripts, if used must have the executable permission.

Debugging the customize script

The output on STDERR while the customize script customize.sh is running goes to nirvana, therefore a simple “set -x” in the sctript customize.sh does not work to get the error messages. To log the output of the customize script to a file, add these commands to the beginning of the script:

exec 1>/data/local/tmp/customize.log 2>&1set -x
Now STDOUT and STDERR output of the script can be found in the file /data/local/tmp/customize.log.

The same technique can be used in the scripts service.sh and post-fs-data.sh (with other files for the output, of course). Note that while post-fs-data.sh is running the data in /data/media/0 is not yet decrypted and can not be used (-> sub directories in /sdcard are also not usable while the script post-fs-data.sh is running).

The section Hints for writing init scripts for Magisk describes a method to automatically disable a Magisk module.

Installing a Magisk Module

While developing a Magisk Module, the Module should be installed in an adb session using the magisk in order to obtain all the output of the installation scripts, the command to install a Magisk Module via the magisk binary is:

magisk --install-module <zip_file_with_the_module>

Example

[ASUS_I006D:/ $ su - -c magisk --install-module /sdcard/Download/syslogd_2.5.0.zip - Current boot slot: _b- Device is system-as-root************************** syslogd by Bernd.Schemmer@gmx.de ********************************************* Powered by Magisk *******************- Extracting module filesThe current environment for this installation isThe version of the installed Magisk is "27.0" (27000)INFO: BOOTMODE is "true" INFO: MODPATH is "/data/adb/modules_update/syslogd"INFO: TMPDIR is "/dev/tmp" INFO: ZIPFILE is "/storage/emulated/0/Download/syslogd_2.5.0.zip"INFO: ARCH is "arm64" INFO: IS64BIT is "true" INFO: API is "34" Installing the Magisk Module with the syslogd "2.5.0" ...Checking the type of the CPU used in this device ....The CPU in this device is a arm64 CPUThe machine type reported by "uname -m" is "aarch64"Correcting the permissions for the new files ...Processing the files in the directory /data/adb/modules_update/syslogd/system/bin ...- DoneASUS_I006D:/ $

The script install_mm.sh can be used to copy the zip file to the phone and install it. It works on a PC with LinuxOS and can also be used on a phone with Android OS.

Usage help for install_mm.sh

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/Scripts ] $ ./install_mm.sh -hinstall_mm.sh v1.0.0 Usage: install_mm.sh [options_for_adb -- ] [--keep] [--force] [--reboot] [magisk_module1|dir1 ... magisk_module#|dir#] Parameter: options_for_adb - options for the adb command; this parameter is only used if running on a PC --keep - do not delete the ZIP file with the Magisk Module (this parameter is only used if the script is running on a PC) --reboot - reboot the phone after installing all Magisk Modules --force - reboot the phone even if not all Magisk Modules could be installed Use the parameter "-h -v" to print the detailed usage helpEnvironment variables used:TARGET_DIR = /sdcard/DownloadADB_OPTIONS = TMPDIR = /tmp[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/Scripts ] $

After the installation of the module, but before the necessary reboot to activate the module, the files for the module are located in the directory

/data/adb/modules_update/<module_name>

After the next reboot Magisk moves these files to the final directory for the Module

/data/adb/modules/<module_name>

The Environment variables defined by Magisk that can be used in the customize script for the Module (customize.sh) are:

#    MAGISK_VER (string): the version string of current installed Magisk (e.g. v20.0)
#    MAGISK_VER_CODE (int): the version code of current installed Magisk (e.g. 20000)
#    BOOTMODE (bool): true if the module is being installed in the Magisk app
#    MODPATH (path): the path where your module files should be installed
#    TMPDIR (path): a place where you can temporarily store files
#    ZIPFILE (path): your module’s installation zip
#    ARCH (string): the CPU architecture of the device. Value is either arm, arm64, x86, or x64
#    IS64BIT (bool): true if $ARCH is either arm64 or x64
#    API (int): the API level (Android version) of the device (e.g. 21 for Android 5.0)

Example values for the environment variables:

INFO: MAGISK_VER is "27.0" INFO: MAGISK_VER_CODE is "27000" INFO: BOOTMODE is "true"INFO: MODPATH is "/data/adb/modules_update/syslogd" INFO: TMPDIR is "/dev/tmp" INFO: ZIPFILE is "/storage/emulated/0/Download/syslogd_2.5.0.zip"INFO: ARCH is "arm64" INFO: IS64BIT is "true" INFO: API is "34"

The environment variable MODPATH is the directory with the files from the Magisk Module while installing the Module: /data/adb/modules_update/<module_name>
-> New files for the directory /system/bin in the Module are in the directory ${MODPATH}/system/bin - that is: /data/adb/modules_update/<module_name>/system/bin.
   (do not use the hardcoded directory name in the script customize.sh)

To cancel the installation of the Magisk Module just delete the directory /data/adb/modules_update/<module_name> before doing the next reboot.

Booting the phone fails after installing a new Magisk Module

If the phone does not finish rebooting after installing a new Magisk module, try connecting to the phone via adb. In most cases this works, as adbd is started very early in the boot process.

If that does not work, either boot the phone from a recovery image with enabled access via adb and mounted /data (like for example TWRP) or reboot the phone into the safe mode to disable all Magisk Modules.

Fixing issues

To fix an issue in an Magisk Module already installed just change the files in the directory /data/adb/modules/<module_name>/system. Changes in the files in that directory should also be visible immediately in the replaced file in /system.

If this is not the case, simply umount and remount the replaced file:

ASUS_I006D:/ $ list_bind_mounts.sh /system/etc/syslog.conf # Checking 1 mount entries .../system/etc/syslog.conf -> /data/adb/modules/syslogd/system/etc/syslog.confASUS_I006D:/ $ ASUS_I006D:/ $ su - -c umount /system/etc/syslog.confASUS_I006D:/ $ su - -c mount --bind /data/adb/modules/syslogd/system/etc/syslog.conf /system/etc/syslog.conf
(... or reboot the phone to activate the changed file)

When everything is working as expected, copy the changes to the source for the Magisk Module and recreate the ZIP file with the module.

The script list_bind_mounts.sh is available here: list_bind_mounts.sh (see also here)

History of this entry

28.08.2024

initial release

How to create logical devices using the device mapper from Android

URL: https://xdaforums.com/t/guide-how-to-create-logical-devices-using-the-device-mapper-from-android.4690991/

How to create logical devices using the device mapper from Android

Note:

All commands in this HowTo must be done as user root.

The current version of Android uses dynamic partitions in the partition super for some of the slot dependent partitions. To use these dynamic partitions, a device mapper is used to create logical devices for them.

The command to work with logical devices in Android is dmctl.

The usage help for dmctl is:

dmctl help

ASUS_I006D:/data/develop/slot_a # dmctl --helpusage: dmctl <command> [command options] dmctl -f filecommands: create <dm-name> [-ro] <targets...> delete <dm-name> list <devices | targets> [-v] getpath <dm-name> getuuid <dm-name> info <dm-name> status <dm-name> resume <dm-name> suspend <dm-name> table <dm-name> help-f file reads command and all parameters from named fileTarget syntax: <target_type> <start_sector> <num_sectors> [target_data]234|ASUS_I006D:/data/develop/slot_a #

dmctl can be used to create, list, and delete logical devices.

To list the existing logical devices on the phone use the command:

dmctl list devices

Example

ASUS_I006D:/ # dmctl list devices Available Device Mapper Devices: com.android.neuralnetworks : 253:19system_ext_b : 253:3userdata : 253:33odm_b : 253:0system_b : 253:2 com.android.media : 253:16 com.android.media.swcodec : 253:11vendor_dlkm_b : 253:5vendor_b : 253:4 com.android.resolv : 253:14 com.android.adservices : 253:31 com.android.mediaprovider : 253:23 com.android.permission : 253:29 com.android.conscrypt : 253:28com.android.art : 253:30 com.android.adbd : 253:12 com.android.tethering : 253:13 com.android.extservices : 253:24 com.android.cellbroadcast : 253:18product_b : 253:1 com.android.wifi : 253:6com.android.uwb : 253:25 com.android.ipsec : 253:10 ASUS_I006D:/ #

Or check the logical devices in the device tree in /dev/block/mapper:

ls -l /dev/block/mapper

ASUS_I006D:/ $ ls -l /dev/block/mapper/ total 0drwxr-xr-x 2 root root 500 2024-08-31 07:55 by-uuidlrwxrwxrwx 1 root root 16 1970-02-16 21:52 com.android.adbd -> /dev/block/dm-12lrwxrwxrwx 1 root root 16 1970-02-16 21:52 com.android.adservices -> /dev/block/dm-31lrwxrwxrwx 1 root root 16 1970-02-16 21:52 com.android.art -> /dev/block/dm-30lrwxrwxrwx 1 root root 16 1970-02-16 21:52 com.android.cellbroadcast -> /dev/block/dm-18lrwxrwxrwx 1 root root 16 1970-02-16 21:52 com.android.conscrypt -> /dev/block/dm-28lrwxrwxrwx 1 root root 16 1970-02-16 21:52 com.android.extservices -> /dev/block/dm-24lrwxrwxrwx 1 root root 16 1970-02-16 21:52 com.android.ipsec -> /dev/block/dm-10lrwxrwxrwx 1 root root 16 1970-02-16 21:52 com.android.media -> /dev/block/dm-16lrwxrwxrwx 1 root root 16 1970-02-16 21:52 com.android.media.swcodec -> /dev/block/dm-11lrwxrwxrwx 1 root root 16 1970-02-16 21:52 com.android.mediaprovider -> /dev/block/dm-23lrwxrwxrwx 1 root root 16 1970-02-16 21:52 com.android.neuralnetworks -> /dev/block/dm-19lrwxrwxrwx 1 root root 16 1970-02-16 21:52 com.android.permission -> /dev/block/dm-29lrwxrwxrwx 1 root root 16 1970-02-16 21:52 com.android.resolv -> /dev/block/dm-14lrwxrwxrwx 1 root root 16 1970-02-16 21:52 com.android.tethering -> /dev/block/dm-13lrwxrwxrwx 1 root root 16 1970-02-16 21:52 com.android.uwb -> /dev/block/dm-25lrwxrwxrwx 1 root root 15 1970-02-16 21:52 com.android.wifi -> /dev/block/dm-6lrwxrwxrwx 1 root root 15 1970-02-16 21:52 odm_b -> /dev/block/dm-0lrwxrwxrwx 1 root root 15 1970-02-16 21:52 product_b -> /dev/block/dm-1lrwxrwxrwx 1 root root 15 1970-02-16 21:52 system_b -> /dev/block/dm-2lrwxrwxrwx 1 root root 15 1970-02-16 21:52 system_ext_b -> /dev/block/dm-3lrwxrwxrwx 1 root root 16 1970-02-16 21:52 userdata -> /dev/block/dm-33lrwxrwxrwx 1 root root 15 1970-02-16 21:52 vendor_b -> /dev/block/dm-4lrwxrwxrwx 1 root root 15 1970-02-16 21:52 vendor_dlkm_b -> /dev/block/dm-5 ASUS_I006D:/ $

To list the dynamic partitions in the partition super the command lpdump can be used.

The usage help for lpdump is:

lpdump --help

ASUS_I006D:/data/develop/slot_a # lpdump --help lpdump - command-line tool for dumping Android Logical Partition images. Usage: lpdump [-s <SLOT#>|--slot=<SLOT#>] [-j|--json] [FILE|DEVICE] Options: -s, --slot=N Slot number or suffix. -j, --json Print in JSON format. -d, --dump-metadata-size Print the space reserved for metadata to stdout in bytes. -a, --all Dump all slots (not available in JSON mode). ASUS_I006D:/data/develop/slot_a #

The command to list the dynamic partitions for the current slot is:

lpdump -s $( getprop ro.boot.slot_suffix ) | grep "super:"
e.g.:

ASUS_I006D:/data/develop/slot_a # lpdump -s $( getprop ro.boot.slot_suffix ) | grep "super:" super: 2048 .. 4584: odm_a (2536 sectors) super: 5120 .. 3411488: product_a (3406368 sectors) super: 3411968 .. 10850648: system_a (7438680 sectors) super: 10851328 .. 11858744: system_ext_a (1007416 sectors) super: 11858944 .. 14465728: vendor_a (2606784 sectors) ASUS_I006D:/data/develop/slot_a #

The command to create a new logical device using dmctl is:

dmctl create [LOGICAL_DEVICE_NAME] linear 0 [PARTITION_SIZE] /dev/block/by-name/super [PARTITION_START_SECTOR]

e.g. the command to create an (additional) device for the logical partition system_b in the partition super listed above is:

ASUS_I006D:/data/develop/slot_a # dmctl create system_a_new linear 0 7438680 /dev/block/by-name/super 3411968 ASUS_I006D:/data/develop/slot_a #
The result is:

ASUS_I006D:/data/develop/slot_a # ls -l /dev/block/mapper/system_a_new lrwxrwxrwx 1 root root 16 2024-09-03 13:29 /dev/block/mapper/system_a_new -> /dev/block/dm-11 ASUS_I006D:/data/develop/slot_a #

To use the new device, mount it - example:

ASUS_I006D:/data/develop/slot_a # mkdir -p /data/local/tmp/system_a_newASUS_I006D:/data/develop/slot_a # # Note: # # The mount parameter "-o ro" is only necessary for logical partitions in the super partition of stock Android ROMs; these partitions can only be mounted in read-only mode#ASUS_I006D:/data/develop/slot_a # mount -o ro /dev/block/mapper/system_a_new /data/local/tmp/system_a_newASUS_I006D:/data/develop/slot_a # ASUS_I006D:/data/develop/slot_a # df -h /data/local/tmp/system_a_newFilesystem       Size Used Avail Use% Mounted on/dev/block/dm-11 3.4G 3.4G   11M 100% /data/local/tmp/system_a_newASUS_I006D:/data/develop/slot_a # ASUS_I006D:/data/develop/slot_a # ls -l /data/local/tmp/system_a_newtotal 136drwxr-xr-x 2 root   root      4096 2009-01-01 01:00 ADFdrwxr-xr-x 2 root   root      4096 2009-01-01 01:00 APDdrwxr-xr-x 2 root   root      4096 2009-01-01 01:00 acctdrwxr-xr-x 2 root   root      4096 2009-01-01 01:00 apexdrwxr-xr-x 2 root   root      4096 2009-01-01 01:00 asdfdrwxr-xr-x 2 root   root      4096 2009-01-01 01:00 batinfo...drwxr-xr-x 2 root   shell     4096 2009-01-01 01:00 vendordrwxr-xr-x 2 root   root      4096 2009-01-01 01:00 vendor_dlkmASUS_I006D:/data/develop/slot_a #

To delete the new logical device use these commands.

# umount the device if still mounted#umount /data/local/tmp/system_a_new   # delete the device#dmctl delete system_a_new

One advantage of the dynamic partition is that these partitions can grow. After a dynamic partition has been enlarged, the list of dynamic partitions in the partition super might look like this:

ASUS_I006D:/ # lpdump -s $( getprop ro.boot.slot_suffix ) | grep "super:"super: 2048 .. 2747392: system_b (2745344 sectors)super: 2747392 .. 3352576: system_ext_b (605184 sectors)super: 3352576 .. 3355320: odm_b (2744 sectors)super: 3355648 .. 4115456: product_b (759808 sectors)super: 4115456 .. 6727000: vendor_b (2611544 sectors)super: 6727680 .. 6728704: product_b (1024 sectors)super: 6728704 .. 6878424: system_b (149720 sectors)super: 6879232 .. 6880968: system_ext_b (1736 sectors)super: 6881280 .. 6884848: product_b (3568 sectors)ASUS_I006D:/ #

In this example the dynamic partition system_b was enlarged:

ASUS_I006D:/ # lpdump -s $( getprop ro.boot.slot_suffix ) | grep "super:" | grep system_bsuper: 2048 .. 2747392: system_b (2745344 sectors)super: 6728704 .. 6878424: system_b (149720 sectors)ASUS_I006D:/ #
To create a logical device for a dynamic partition with one or more extends the parameter "-f" of the dcmtl command must be used.

E.g. the commands to create a logical device for the dynamic partition system_b in the list for the partition super above are:

# create an input file with the necessary commands to create the logical device for dmctl # cat >/data/local/tmp/dmctl_input <<EOTcreate dm-b-system_b    linear 0 2745344 /dev/block/by-name/super 2048   linear 2745344 149720 /dev/block/by-name/super 6728704EOT# create the logical device using the commands from the input file #dmctl -f /data/local/tmp/dmctl_input

The result is:

ASUS_I006D:/data/local/tmp # ls -l /dev/block/mapper/dm-b-system_blrwxrwxrwx 1 root root 16 2024-09-03 13:53 /dev/block/mapper/dm-b-system_b -> /dev/block/dm-10ASUS_I006D:/data/local/tmp # # mount the new logical device #ASUS_I006D:/data/local/tmp # mkdir -p /data/local/tmp/system_bASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # mount -o ro /dev/block/mapper/dm-b-system_b /data/local/tmp/system_bASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # df -h /Filesystem      Size Used Avail Use% Mounted on/dev/block/dm-0 1.3G 1.3G 4.2M 100% /system/usr/share/zoneinfo/tzdataASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # df -h /data/local/tmp/system_bFilesystem       Size Used Avail Use% Mounted on/dev/block/dm-10 1.3G 1.3G 4.2M 100% /data/local/tmp/system_bASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # ls -l /data/local/tmp/system_btotal 140drwxr-xr-x 2 root   root      4096 2009-01-01 01:00 ADFdrwxr-xr-x 2 root   root      4096 2009-01-01 01:00 APDdrwxr-xr-x 2 root   root      4096 2009-01-01 01:00 acctdrwxr-xr-x 2 root   root      4096 2009-01-01 01:00 apex...drwxr-xr-x 2 root   root      4096 2009-01-01 01:00 vendor_dlkmASUS_I006D:/data/local/tmp #

The command dmctl table can be used to check whether the configuration for a new device is correct.

For example, to make sure, that the new logical device created for the dynamic partition system_b is equal to the original device for the dynamic partition system_b compare the output of the command dmctl table for both logical devices:

ASUS_I006D:/data/local/tmp # dmctl table system_bTargets in the device-mapper table for system_b:0-2745344: linear, 259:3 20482745344-2895064: linear, 259:3 6728704ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # dmctl table dm-b-system_bTargets in the device-mapper table for dm-b-system_b:0-2745344: linear, 259:3 20482745344-2895064: linear, 259:3 6728704ASUS_I006D:/data/local/tmp #

I created a simple shell script for Android to show the usage of the dmctl command

remount_dynamic_partitions.sh

The script creates and mounts an additional logical device for each dynamic partition in the super partition for the current slot. This is pretty useless, it only serves as a proof of concept.

The usage for the script is:

remount_dynamic_partitions.sh -v -h

1|ASUS_I006D:/ # /data/local/tmp/remount_dynamic_partitions.sh -h -v remount_dynamic_partitions.sh 1.1.0 - create new additional logical devices for the dynamic partitions in the Android super partition and mount them Usage: remount_dynamic_partitions.sh [var=value] [--verbose|-v] [--help|-h] [--keep|-k] Parameter: --verbose | -v print more messages --help | -h print this usage help --keep | -k do not delete the input files for dmctl The options to mount the partitions are read from the environment variable MOUNT_OPTIONS, the default mount options are "-o ro" (=> mount read-only). To mount the partitions read/write set the variable MOUNT_OPTIONS to "-o rw" before starting the script; to not use any mount option set the variable MOUNT_OPTIONS to "none" before starting the script The environment variables supported by this script are: MOUNT_OPTIONS; the current value is "-o ro" BASEDIR; the current value is "/data/local/tmp" LPDUMP; the current value is "/system/bin/lpdump" DMCTL; the current value is "/system/bin/dmctl" AWK; the current value is "/system/bin/awk" TMPDIR; the current value is "/data/local/tmp" SLOT; the current value is "_b" DELETE_SCRIPT_NAME; the current value is "/data/local/tmp/delete_dm_devices_b.6259.sh" UMOUNT_SCRIPT_NAME; the current value is "/data/local/tmp/umount_dm_devices_b.6259.sh" DEVICE_NAME_PREFIX; the current value is "dm-b-" 1|ASUS_I006D:/ #

---

An example output of the script looks like this:

Example output of the script remount_dynamic_partitions.sh

ASUS_I006D:/data/local/tmp # /data/local/tmp/remount_dynamic_partitions.shremount_dynamic_partitions.sh 1.0.0 - create new additional logical devices for the dynamic partitions in the Android super partition and mount them Retrieving the list of logical partitions for the slot "_b" in the super partition ... The logical partitions for the slot "_b" in the super partition are: odm_b product_b system_b system_ext_b vendor_b Mounting the volumes in the super partition for the slot "_b" to sub directories in the directory "/data/local/tmp/b" ... The mount options used are "-o ro" # Processing the logical partition "odm_b" ... The mount point for the logical partition "odm_b" is "/data/local/tmp/b/odm_b" Creating the DM mapper device "dm-b-odm_b" for the logical partition "odm_b" ... # Processing the logical partition "product_b" ... The mount point for the logical partition "product_b" is "/data/local/tmp/b/product_b" Creating the DM mapper device "dm-b-product_b" for the logical partition "product_b" ... # Processing the logical partition "system_b" ... The mount point for the logical partition "system_b" is "/data/local/tmp/b/system_b" The DM mapper device "dm-b-system_b" already exists # Processing the logical partition "system_ext_b" ... The mount point for the logical partition "system_ext_b" is "/data/local/tmp/b/system_ext_b" Creating the DM mapper device "dm-b-system_ext_b" for the logical partition "system_ext_b" ... # Processing the logical partition "vendor_b" ... The mount point for the logical partition "vendor_b" is "/data/local/tmp/b/vendor_b" Creating the DM mapper device "dm-b-vendor_b" for the logical partition "vendor_b" ... Creating the script to umount the mount points for the new devices "/data/local/tmp/umount_dm_devices_b.32518.sh" ... -rwxr-xr-x 1 root root 638 2024-09-03 14:42 /data/local/tmp/umount_dm_devices_b.32518.sh Creating the script to delete the temporary devices "/data/local/tmp/delete_dm_devices_b.32518.sh" ... -rwxr-xr-x 1 root root 1044 2024-09-03 14:42 /data/local/tmp/delete_dm_devices_b.32518.sh Use /data/local/tmp/umount_dm_devices_b.32518.sh yes ; /data/local/tmp/delete_dm_devices_b.32518.sh yes to umount and delete the temporary devices Mounted logical partitions are: /dev/block/dm-15 1.2M 1.2M 4.0K 100% /data/local/tmp/b/odm_b /dev/block/dm-16 367M 366M 1.1M 100% /data/local/tmp/b/product_b /dev/block/dm-10 1.3G 1.3G 4.2M 100% /data/local/tmp/b/system_b /dev/block/dm-18 291M 290M 908K 100% /data/local/tmp/b/system_ext_b /dev/block/dm-19 1.2G 1.2G 3.8M 100% /data/local/tmp/b/vendor_b ASUS_I006D:/data/local/tmp #

---

There is no plain command to get the backend used for the logical devices. The commands to get the backend used for a logical device are:

ASUS_I006D:/data/local/tmp # dmctl getpath system_a /dev/block/dm-2ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # ls /sys/block/dm-2/slaves/ sda19ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # ls -l /dev/block/by-name/ | grep sda19 lrwxrwxrwx 1 root root 16 1970-02-21 06:39 super -> /dev/block/sda19 ASUS_I006D:/data/local/tmp #-> The backend for the logical device system_a is the partition sda19 and that is the super partition.

I wrote a little script to print the backends for the logical devices

list_logical_device_backends.sh

The usage for the script is:

list_logical_device_backends.sh -v -h

ASUS_I006D:/data/local/tmp # ./list_logical_device_backends.sh -v -h list_logical_device_backends.sh 1.1.0 - list the backends for logical devices Usage: list_logical_device_backends.sh [var=value] [--verbose|-v] [--help|-h] [--noheader] [--noloop] [logical_device# ...] Parameter: --verbose | -v print more messages --help | -h print this usage help --noheader print only the results --noloop ignore loop devices The environment variables supported by this script are: DMCTL; the current value is "/system/bin/dmctl" 1|ASUS_I006D:/data/local/tmp #

The output of the script looks like this:

Example output of the script list_logical_device_backends.sh

ASUS_I006D:/data/local/tmp # ./list_logical_device_backends.shlist_logical_device_backends.sh 1.0.0 - list the backends for logical devices Retrieving the list of logical devices ... # Logical device = dm device [ -> backend_device ] -> backend device [= symbolic name for backend device] # or # Logical device = dm device -> loop device = backend file for the loop device vendor_b-cow = dm-9 -> sda19 = "super" vendor_b = dm-4 -> sda19 = "super" userdata = dm-42 -> sda23 = "userdata" system_ext_b-cow = dm-6 -> sda19 = "super" system_ext_b = dm-1 -> sda19 = "super" system_b-cow = dm-5 -> sda19 = "super" system_b = dm-0 -> sda19 = "super" product_b-cow = dm-8 -> sda19 = "super" product_b = dm-3 -> sda19 = "super" odm_b-cow = dm-7 -> sda19 = "super" odm_b = dm-2 -> sda19 = "super" com.android.wifi = dm-29 -> loop21 = /data/apex/decompressed/com.android.wifi@350090000.decompressed.apex com.android.uwb = dm-41 -> loop15 = /data/apex/decompressed/com.android.uwb@350090000.decompressed.apex com.android.tethering = dm-38 -> loop19 = /data/apex/decompressed/com.android.tethering@350090000.decompressed.apex com.android.scheduling = dm-28 -> loop34 = /data/apex/decompressed/com.android.scheduling@350090000.decompressed.apex com.android.resolv = dm-37 -> loop18 = /data/apex/decompressed/com.android.resolv@350090000.decompressed.apex com.android.permission = dm-24 -> loop24 = /data/apex/decompressed/com.android.permission@350090000.decompressed.apex com.android.ondevicepersonalization = dm-12 -> loop31 = /data/apex/decompressed/com.android.ondevicepersonalization@350090000.decompressed.apex com.android.neuralnetworks = dm-13 -> loop30 = /data/apex/decompressed/com.android.neuralnetworks@350090000.decompressed.apex com.android.mediaprovider = dm-39 -> loop17 = /data/apex/decompressed/com.android.mediaprovider@350090000.decompressed.apex com.android.media.swcodec = dm-32 -> loop23 = /data/apex/decompressed/com.android.media.swcodec@350090000.decompressed.apex com.android.media = dm-33 -> loop22 = /data/apex/decompressed/com.android.media@350090000.decompressed.apex com.android.ipsec = dm-11 -> loop33 = /data/apex/decompressed/com.android.ipsec@350090000.decompressed.apex com.android.extservices = dm-40 -> loop16 = /data/apex/decompressed/com.android.extservices@350090000.decompressed.apex com.android.conscrypt = dm-25 -> loop4 = /data/apex/decompressed/com.android.conscrypt@350090000.decompressed.apex com.android.configinfrastructure = dm-17 -> loop27 = /data/apex/decompressed/com.android.configinfrastructure@350090000.decompressed.apex com.android.cellbroadcast = dm-14 -> loop32 = /data/apex/decompressed/com.android.cellbroadcast@350090000.decompressed.apex com.android.art = dm-27 -> loop26 = /data/apex/decompressed/com.android.art@350090000.decompressed.apex com.android.appsearch = dm-23 -> loop25 = /data/apex/decompressed/com.android.appsearch@350090000.decompressed.apex com.android.adservices = dm-20 -> loop28 = /data/apex/decompressed/com.android.adservices@350090000.decompressed.apex com.android.adbd = dm-31 -> loop20 = /data/apex/decompressed/com.android.adbd@350090000.decompressed.apex ASUS_I006D:/data/local/tmp $

The script

list_logical_device_usage.sh

can be used to list the logical partitions defined on a physical partition

The usage for the script is:

list_logical_device_usage.sh -v -h

ASUS_I006D:/data/local/tmp # ./list_logical_device_usage.sh -v -h list_logical_device_usage.sh 1.0.0 - list logical devices configured on a physical device Usage: list_logical_device_usage.sh [var=value] [--verbose|-v] [--help|-h] [--short] [--noheader] [physical_device] Parameter: --verbose | -v print more messages --help | -h print this usage help --short do not print infos about unknown disk parts --noheader print only the results "physical_device" is the name of the physical device, example: "sda19", "/dev/block/sda19", or "super" The default physical device is "super" The environment variables supported by this script are: DMCTL; the current value is "/system/bin/dmctl" TMPFILE; the current value is "/data/local/tmp/tmpfile.30603" 1|ASUS_I006D:/data/local/tmp #

The output of the script looks like this:

Example output of the script list_logical_device_usage.sh

ASUS_I006D:/data/local/tmp $ ./list_logical_device_usage.sh list_logical_device_usage.sh 1.0.0 - list logical devices configured on a physical device Retrieving the logical device usage map for the device /dev/block/sda19 ( = super ) ... Logical device usage map for the device /dev/block/sda19 ( = super ) Start Length End Used for the Logical Device ------------------------------------------------------------- 2048 2536 4584 odm_a 4584 536 5120 unknown usage 5120 3406368 3411488 product_a 3411488 480 3411968 unknown usage 3411968 7438680 10850648 system_a 10850648 680 10851328 unknown usage 10851328 1007416 11858744 system_ext_a 11858744 200 11858944 unknown usage 11858944 2606784 14465728 vendor_a ASUS_I006D:/data/local/tmp $

Notes

This post only covers the basic functions of the device mapper from Android. The device mapper can do much more, but unfortunately there is no really useful documentation for the tool.

The binary dmctl is part of the Android OS; in most Android OS versions the binary lpdump is also part of the OS.

If not:

The binary lpdump is part of the RO2RW tools that can be downloaded from here: https://sourceforge.net/projects/multi-function-patch/
lpdump is also part of the Magisk Module ro2rw_3.7.3.0.zip (see also the list of available Magisk Modules on this website below)

see also this page https://www.programmersought.com/article/962810691772/ for more details about dynamic partition usage in Android

see also https://source.android.com/docs/core/ota/dynamic_partitions/implement

dmctl also works when the phone is booted from the TWRP recovery image. The binary dmctl is not part of the TWRP image, so either the dmctl binary must be copied to the phone or the volume system must be mounted while booted into TWRP.

The sourcecode for dmctl is part of the TWRP repository (in the directory ./system/core/fs_mgr/tools) but I have not scceeded in building a TWRP image that includes the dmctl binary

for the records: As of 05.09.2024, I have not succeeded in creating logical partitions for the dynamic partitions in the inactive slot with dmctl.

see How to create a new dynamic partition in the partition super for a more useful usage example

see How to access the systems partition while the phone is booted into the LineageOS recovery for an example for using these commands in the LineageOS recovery to mount the system partitions

History of this entry

05.09.2024

initial release

07.09.2024

updated the usage help for the scripts list_logical_device_backends.sh and remount_dynamic_partitions.sh

How to create a new dynamic partition in the partition super

URL : https://xdaforums.com/t/guide-how-to-create-a-new-dynamic-partition-in-the-partition-super.4691372/

How to create a new dynamic partition in the partition super

Please note that the documentation on dynamic partitions and the associated tools in the Android documentation is very poor and many points (such as command parameters) are not documented at all (or at least I have not found the documentation).
For this reason, the instructions in this HowTo may be incorrect or incomplete. They worked in my environment (see test environments below), but they may not work in other environments.

If you want to try out the instructions on your phone, it's strongly recommended to first create a backup of the data on your phone.

In most Android disk configurations, the space in the super partition in Android is only used partially and there is still free space to create additional partitions.

In this post I describe the steps necessary to create additional dynamic partitions in the super partition.

You might ask why you should do this? Well; it's fun to get it working and, more important, the data on additional dynamic partitions survive OS upgrades and also factory resets.

Notes

Most of the commands in this HowTo require access as user root.

The notes at the end of this section will tell you how to get the necessary tools for the instructions.

Before adding new dynamic partitions to the super partition it's recommended to check (and document) the current configuration for dynamic partitions. This can be done using the tool lpdump,

Example:

ASUS_I006D:/ $ lpdump -s $( getprop ro.boot.slot_suffix ) | grep "super:" super: 2048 .. 4584: odm_a (2536 sectors)super: 5120 .. 3411264: product_a (3406144 sectors)super: 3411968 .. 10851264: system_a (7439296 sectors)super: 10851328 .. 10875936: system_a (24608 sectors)super: 10876928 .. 11641072: system_ext_a (764144 sectors)super: 11641856 .. 11858880: vendor_a (217024 sectors)super: 11858944 .. 14247288: vendor_a (2388344 sectors)ASUS_I006D:/ $

It is also important to check the free space on the super partition. I don't know of a reliable way to determine this value, so I recommend using the output of lpdump and doing some manual calculations.
And be aware that OS updates may need to increase one or more dynamic partitions in the super partition to accommodate all the files for a new OS version. Therefore, do not use all the free space in the super partition if you plan to install OS upgrades in the future.

The tool to create another dynamic partition in Android is lpadd; the usage syntax for lpadd is:

lpadd [options] SUPER PARTNAME PARTGROUP [IMAGE]

There is no parameter for lpadd to define the size for the new partition. To define the size for the new partition the parameter IMAGE is used: IMAGE is the name of a file that is copied to the new partition; this can be any file or block device. The size of the new partition is the size of the image file or block device used for the parameter IMAGE.

Without the parameter IMAGE, lpadd creates a dynamic partition with zero length.

For example to create a new logical partition with the size of one of the existing partitions just use that partition for the parameter IMAGE, example:

To create a new dynamic partition with the size of the partition boot_a (= 100 MB) execute :

lpadd /dev/block/by-name/super <partition_name> default /dev/block/by-name/boot_a

Example:

ASUS_I006D:/ # lpadd /dev/block/by-name/super new_part001 default /dev/block/by-name/boot_aWriting data for partition new_part001...Done.ASUS_I006D:/ # ASUS_I006D:/ # lpdump -s $( getprop ro.boot.slot_suffix ) | grep "super:" super: 2048 .. 4584: odm_a (2536 sectors)super: 5120 .. 3411264: product_a (3406144 sectors)super: 3411968 .. 10851264: system_a (7439296 sectors)super: 10851328 .. 10875936: system_a (24608 sectors)super: 10876928 .. 11641072: system_ext_a (764144 sectors)super: 11641856 .. 11858880: vendor_a (217024 sectors)super: 11858944 .. 14247288: vendor_a (2388344 sectors)super: 14247936 .. 14444544: new_part001 (196608 sectors)ASUS_I006D:/ # ASUS_I006D:/ # blockdev --getsize /dev/block/by-name/boot_a196608ASUS_I006D:/ #

To use this new dynamic partition, create a logical device for it using dmctl:

# get the values for the dmctl command from the output of the lpdump command:#ASUS_I006D:/ # lpdump -s $( getprop ro.boot.slot_suffix ) | grep "super:" | grep new_part001super: 14247936 .. 14444544: new_part001 (196608 sectors)ASUS_I006D:/ # ASUS_I006D:/ # dmctl create newpart001 linear 0 196608 /dev/block/by-name/super 14247936ASUS_I006D:/ # # check the result:#ASUS_I006D:/ # dmctl list devicesAvailable Device Mapper Devices:system_ext_a : 253:3userdata : 253:40odm_a : 253:0system_a : 253:2vendor-verity : 253:8vendor_a : 253:4system_ext-verity : 253:6newpart001 : 253:10odm-verity : 253:9product-verity : 253:7system-verity : 253:5com.android.wifi : 253:11com.android.uwb : 253:19product_a : 253:1ASUS_I006D:/ # ASUS_I006D:/ # ls -l /dev/block/mapper/newpart001lrwxrwxrwx 1 root root 16 2024-09-06 15:24 /dev/block/mapper/newpart001 -> /dev/block/dm-10ASUS_I006D:/ #

Next create a filesystem on the new device, e.g:

ASUS_I006D:/ # mkfs.ext3 /dev/block/dm-10mke2fs 1.46.6 (1-Feb-2023)Discarding device blocks: doneCreating filesystem with 24576 4k blocks and 24576 inodesAllocating group tables: doneWriting inode tables: doneCreating journal (1024 blocks): doneWriting superblocks and filesystem accounting information: doneASUS_I006D:/ #

And finally create a mount point and mount the new filesystem:

ASUS_I006D:/ # mkdir -p /data/local/tmp/aASUS_I006D:/ # ASUS_I006D:/ # chmod 1777 /data/local/tmp/aASUS_I006D:/ # ASUS_I006D:/ # /data/adb/magisk/busybox chcon -R -v --reference=/data/local/tmp /data/local/tmp/aASUS_I006D:/ # ASUS_I006D:/ # mount /dev/block/dm-10 /data/local/tmp/aASUS_I006D:/ # ASUS_I006D:/ # ls -al /data/local/tmp/atotal 23drwxr-xr-x 3 root root 4096 2024-09-06 15:27 .drwxrwx--x 6 shell shell 3452 2024-09-06 15:28 ..drwx------ 2 root root 16384 2024-09-06 15:27 lost+foundASUS_I006D:/ # ASUS_I006D:/ # df -h /data/local/tmp/aFilesystem Size Used Avail Use% Mounted on/dev/block/dm-10 86M 28K 86M 1% /data/local/tmp/aASUS_I006D:/ # ASUS_I006D:/ # touch /data/local/tmp/a/Was_so_alles_geht_in_AndroidASUS_I006D:/ # ASUS_I006D:/ # ls -al /data/local/tmp/atotal 23drwxr-xr-x 3 root root 4096 2024-09-06 15:31 .drwxrwx--x 6 shell shell 3452 2024-09-06 15:28 ..-rw-r--r-- 1 root root 0 2024-09-06 15:31 Was_so_alles_geht_in_Androiddrwx------ 2 root root 16384 2024-09-06 15:27 lost+foundASUS_I006D:/ #

The new dynamic partition survives a reboot (of course ...); the logical device for the new dynamic partition is created automatically by the Android OS after a reboot, so the only necessary command after a reboot to use the new device is the mount command.

This could be done, for example in a Magisk init script, example:

cat /data/adb/service.d/mount_my_new_device.sh

ASUS_I006D:/ # cat /data/adb/service.d/mount_my_new_device.sh # for debugging # exec 1>/data/cache/${0##*/}.log 2>&1 set -x DM_DEVICE="/dev/block/mapper/new_part001" MOUNT_POINT="/data/local/tmp/a" i=0 max=120 echo "Waiting up to $max seconds for the device \"${DM_DEVICE}\" ..." while [ $i -lt $max ] ; do if [ -b /dev/block/mapper/new_part001 ] ; then echo "The device \"${DM_DEVICE}\" is available after $i second(s)" # create the mount point if missing # if [ ! -d i"${MOUNT_POINT}" ] ; then mkdir -p "${MOUNT_POINT}" chmod 777 "${MOUNT_POINT}" /data/adb/magisk/busybox chcon -R -v --reference=/data/local/tmp /data/local/tmp/a fi mount "${DM_DEVICE}" "${MOUNT_POINT}" echo df -h echo break fi (( i = i + 1 )) sleep 1 doneASUS_I006D:/ #

To create a new dynamic partition with another size just create a temporary file that you can use for the parameter IMAGE for the lpadd command (see below for an example)

To delete a dynamic partition the command lpmake can be used ... but this command rewrites the complete partition table for the partition super and requires a lot of parameter ( ... with correct values ...). Therefor I recommend not to use that command.

It's more simple to delete a dynamic partition while the phone is booted into the fastboot mode; the command to boot the phone into the fastboot mode is:

adb reboot fastboot

To list the dynamic partitions in fastboot mode use the command:

fastboot getvar all 2>&1 | grep "is-logical:" | grep ":yes"

Example:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts ] $ fastboot getvar all 2>&1 | grep "is-logical:" | grep ":yes"(bootloader) is-logical:odm_a:yes(bootloader) is-logical:product_a:yes(bootloader) is-logical:system_a:yes(bootloader) is-logical:system_ext_a:yes(bootloader) is-logical:vendor_a:yes(bootloader) is-logical:new_part001:yes(bootloader) is-logical:new_part002:yes[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts ] $

To delete a dynamic partition while in fastboot mode use the command:

fastboot delete-logical-partition <partition_name>

Example:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts ] $ fastboot delete-logical-partition new_part002Deleting 'new_part002' OKAY [ 0.008s]Finished. Total time: 0.008s[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts ] $# Check the result:[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts ] $ fastboot getvar all 2>&1 | grep "is-logical:" | grep ":yes"(bootloader) is-logical:odm_a:yes(bootloader) is-logical:product_a:yes(bootloader) is-logical:system_a:yes(bootloader) is-logical:system_ext_a:yes(bootloader) is-logical:vendor_a:yes(bootloader) is-logical:new_part001:yes[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts ] $ In fastboot mode it's also possible to increase the size of a dynamic partition using the command

fastboot resize-logical-partition <partition_name> <new_size_in_bytes>

(Note that you nust increase the filesystem on the device from within the Android OS to use the new space)

Or create a new dynamic partition:

fastboot create-logical-partition <partition_name> <partition_size>

When done, exit the fastboot mode and boot the phone into the Android OS again:

fastboot reboot

Note that fastboot uses the "holes" in the super partition to create a new dynamic partition, example:

Example for creating a dynamic partition in fastboot mode

Logical partition map for the physical partition super before creating a new dynamic partition in fastboot mode:

ASUS_I006D:/data/local/tmp # ./list_logical_device_usage.sh list_logical_device_usage.sh 1.0.0 - list logical devices configured on a physical device Retrieving the logical device usage map for the device /dev/block/sda19 ( = super ) ... Logical device usage map for the device /dev/block/sda19 ( = super ) Start      Length     End        Used for the Logical Device-------------------------------------------------------------2048       2536       4584       odm_a4584       536        5120       unknown usage5120       3406144    3411264    product_a3411264    704        3411968    unknown usage3411968    7439296    10851264   system_a10851264   64         10851328   unknown usage10851328   24608      10875936   system_a10875936   992        10876928   unknown usage10876928   764144     11641072   system_ext_a11641072   784        11641856   unknown usage11641856   217024     11858880   vendor_a11858880   64         11858944   unknown usage11858944   2388344    14247288   vendor_a14247288   648        14247936   unknown usage14247936   196608     14444544   new_part00114444544   102400     14546944   new_part002 ASUS_I006D:/data/local/tmp #

Create a new dynamic partition with 2 GB while the phone is booted into fastboot:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts ] $ fastboot create-logical-partition new_part003 2606784Creating 'new_part003'                             OKAY [ 0.008s] Finished. Total time: 0.008s[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/scripts ] $

and reboot the phone into the Android OS:

fastboot reboot

Logical device usage map for the partition super after creating the new dynamic partition in fastboot mode:

ASUS_I006D:/data/local/tmp # ./list_logical_device_usage.sh list_logical_device_usage.sh 1.0.0 - list logical devices configured on a physical device Retrieving the logical device usage map for the device /dev/block/sda19 ( = super ) ... Logical device usage map for the device /dev/block/sda19 ( = super ) Start      Length     End        Used for the Logical Device-------------------------------------------------------------2048       2536       4584       odm_a4584       472        5056       unknown usage5056       64         5120       new_part0035120       3406144    3411264    product_a3411264    640        3411904    unknown usage3411904    64         3411968    new_part0033411968    7439296    10851264   system_a10851264   64         10851328   new_part00310851328   24608      10875936   system_a10875936   928        10876864   unknown usage10876864   64         10876928   new_part00310876928   764144     11641072   system_ext_a11641072   720        11641792   unknown usage11641792   64         11641856   new_part00311641856   217024     11858880   vendor_a11858880   64         11858944   new_part00311858944   2388344    14247288   vendor_a14247288   584        14247872   unknown usage14247872   64         14247936   new_part00314247936   196608     14444544   new_part00114444544   102400     14546944   new_part00214546944   960        14547904   unknown usage14547904   4648       14552552   new_part003 ASUS_I006D:/data/local/tmp #

After the reboot, the Android OS automatically creates a new logical device for the new dynamic partition:

ASUS_I006D:/data/local/tmp # dmctl list devices | grep new_part003new_part003          : 253:7 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # dmctl table new_part003Targets in the device-mapper table for new_part003: 0-64: linear, 259:3 5056 64-128: linear, 259:3 3411904 128-192: linear, 259:3 10851264 192-256: linear, 259:3 10876864 256-320: linear, 259:3 11641792 320-384: linear, 259:3 11858880 384-448: linear, 259:3 14247872 448-5096: linear, 259:3 14547904 ASUS_I006D:/data/local/tmp #

And now to the real cool things:

Create a new dynamic partition with the size of 50 MB

ASUS_I006D:/data/local/tmp # dd if=/dev/zero of=/data/local/tmp/50mb_file bs=50m count=1 1+0 records in 1+0 records out 52428800 bytes (50 M) copied, 0.074281 s, 673 M/sASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # du -hs /data/local/tmp/50mb_file 50M /data/local/tmp/50mb_file ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # /system/bin/ro2rw/lpadd /dev/block/by-name/super new_part001 default /data/local/tmp/50mb_file Writing data for partition new_part001... Done. ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # lpdump | grep super: | grep new_part001super: 9566208 .. 9668608: new_part001 (102400 sectors) ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # dmctl create new_part001 linear 0 102400 /dev/block/by-name/super 9566208 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # dmctl table new_part001Targets in the device-mapper table for new_part001: 0-102400: linear, 259:3 9566208 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # mkfs.ext3 /dev/block/mapper/new_part001 mke2fs 1.46.2 (28-Feb-2021)Discarding device blocks: done Creating filesystem with 12800 4k blocks and 12800 inodes Allocating group tables: done Writing inode tables: done Creating journal (1024 blocks): doneWriting superblocks and filesystem accounting information: done ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # mkdir -p /data/local/tmp/a ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # mount /dev/block/mapper/new_part001 /data/local/tmp/a ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # df -h /data/local/tmp/aFilesystem Size Used Avail Use% Mounted on/dev/block/dm-10 44M 24K 44M 1% /data/local/tmp/a ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # touch /data/local/tmp/a/working ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # ls -l /data/local/tmp/a/working-rw-r--r-- 1 root root 0 2024-09-06 20:11 /data/local/tmp/a/working ASUS_I006D:/data/local/tmp #

Check the current config (the new dynamic partition is the partition "new_part001" in this example and it's mounted to /data/local/tmp/a)

ASUS_I006D:/data/local/tmp # getprop ro.boot.slot_suffix_bASUS_I006D:/data/local/tmp # # The running OS on the phone is the original Android 13 from ASUS for the ASUS Zenfone 8 #ASUS_I006D:/data/local/tmp # getprop ro.vendor.build.fingerprintasus/EU_I006D/ASUS_I006D:11/RKQ1.210303.002/33.1004.0610.60:user/release-keysASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # lpdump -s 1 | grep "^super:"super: 2048 .. 4256: odm_a (2208 sectors)super: 6144 .. 3898152: product_a (3892008 sectors)super: 3899392 .. 5883584: system_a (1984192 sectors)super: 5883904 .. 6957264: system_ext_a (1073360 sectors)super: 6959104 .. 9564928: vendor_a (2605824 sectors)super: 9566208 .. 9668608: new_part001 (102400 sectors)ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # lpdump -s 0 | grep "^super:"super: 2048 .. 4256: odm_a (2208 sectors)super: 6144 .. 3898152: product_a (3892008 sectors)super: 3899392 .. 5883584: system_a (1984192 sectors)super: 5883904 .. 6957264: system_ext_a (1073360 sectors)super: 6959104 .. 9564928: vendor_a (2605824 sectors)super: 9566208 .. 9668608: new_part001 (102400 sectors)ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # echo "This partition was created while Android OS $( getprop ro.vendor.build.fingerprint ) was running" >/data/local/tmp/a/VERSIONASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # cat /data/local/tmp/a/VERSIONThis partition was created while Android OS asus/EU_I006D/ASUS_I006D:11/RKQ1.210303.002/33.1004.0610.60:user/release-keys was runningASUS_I006D:/data/local/tmp #

Now we update the OS -- this may take some time ... aber ich habe da mal was vorbereitet:

The config after the OS update with factory reset is:

# This is still the Android 13 from ASUS but another version (and in the other slot, of course ...)
#
ASUS_I006D:/ $ getprop ro.vendor.build.fingerprintasus/EU_I006D/ASUS_I006D:11/RKQ1.210303.002/33.1004.0610.60:user/release-keysASUS_I006D:/ $ ASUS_I006D:/ $ getprop ro.boot.slot_suffix_aASUS_I006D:/ $ ASUS_I006D:/ $ getprop ro.vendor.build.fingerprintasus/EU_I006D/ASUS_I006D:11/RKQ1.210303.002/33.1004.0610.60:user/release-keysASUS_I006D:/ $ ASUS_I006D:/ $ lpdump -s 0 | grep "^super:"super: 2048 .. 4584: odm_a (2536 sectors)super: 5120 .. 3411488: product_a (3406368 sectors)super: 3411968 .. 9566208: system_a (6154240 sectors)super: 9566208 .. 9668608: new_part001 (102400 sectors)super: 9668608 .. 10953048: system_a (1284440 sectors)super: 10953728 .. 11961144: system_ext_a (1007416 sectors)super: 11961344 .. 14568128: vendor_a (2606784 sectors)ASUS_I006D:/ $ ASUS_I006D:/ $ lpdump -s 1 | grep "^super:"super: 2048 .. 4256: odm_a (2208 sectors)super: 6144 .. 3898152: product_a (3892008 sectors)super: 3899392 .. 5883584: system_a (1984192 sectors)super: 5883904 .. 6957264: system_ext_a (1073360 sectors)super: 6959104 .. 9564928: vendor_a (2605824 sectors)super: 9566208 .. 9668608: new_part001 (102400 sectors)ASUS_I006D:/ $ ASUS_I006D:/ # mkdir -p /data/local/tmp/aASUS_I006D:/ $ ASUS_I006D:/ # chmod 1777 /data/local/tmp/aASUS_I006D:/ # ASUS_I006D:/ # /data/adb/magisk/busybox chcon -R -v --reference=/data/local/tmp /data/local/tmp/acontext of /data/local/tmp/a retained as u:object_r:shell_data_file:s0ASUS_I006D:/ #ASUS_I006D:/ # mount /dev/block/mapper/new_part001 /data/local/tmp/a
ASUS_I006D:/ # ASUS_I006D:/ # df -h /data/local/tmp/aFilesystem      Size Used Avail Use% Mounted on/dev/block/dm-0 44M 32K   44M   1% /data/local/tmp/aASUS_I006D:/ # ASUS_I006D:/ # ls -l /data/local/tmp/atotal 28-rw-r--r-- 1 root root   134 2024-09-06 20:14 VERSIONdrwx------ 2 root root 16384 2024-09-06 20:11 lost+found-rw-r--r-- 1 root root     0 2024-09-06 20:11 workingASUS_I006D:/ # ASUS_I006D:/ # cat /data/local/tmp/a/VERSIONThis partition was created while Android OS asus/EU_I006D/ASUS_I006D:11/RKQ1.210303.002/33.1004.0610.60:user/release-keys was runningASUS_I006D:/ #

-> The new dynamic partition with the data is still there

Next we update a file in the new dynamic partition and install another ROM:
ASUS_I006D:/ # echo "The partition is still there after OS upgrade to $( getprop ro.vendor.build.fingerprint ) " >> /data/local/tmp/a/VERSIONASUS_I006D:/ # ASUS_I006D:/ # cat /data/local/tmp/a/VERSIONThis partition was created while Android OS asus/EU_I006D/ASUS_I006D:11/RKQ1.210303.002/33.1004.0610.60:user/release-keys was runningThe partition is still there after OS upgrade to asus/EU_I006D/ASUS_I006D:11/RKQ1.210303.002/33.1004.0610.60:user/release-keysASUS_I006D:/ #

Now we install OmniROM 14 (that is Android 14) and perform a factory reset .

The config after the installation of OmniROM 14 with factory reset is:

ASUS_I006D:/ # getprop ro.boot.slot_suffix_bASUS_I006D:/ #ASUS_I006D:/ # getprop ro.vendor.build.fingerprintasus/omni_zenfone8/zenfone8:14/AP2A.240805.005/eng.xtrnaw.20240808.122933:user/release-keysASUS_I006D:/ # ASUS_I006D:/ # lpdump -s 0 | grep "^super:"super: 2048 .. 4584: odm_a (2536 sectors)super: 5120 .. 3411488: product_a (3406368 sectors)super: 3411968 .. 9566208: system_a (6154240 sectors)super: 9566208 .. 9668608: new_part001 (102400 sectors)super: 9668608 .. 10953048: system_a (1284440 sectors)super: 10953728 .. 11961144: system_ext_a (1007416 sectors)super: 11961344 .. 14568128: vendor_a (2606784 sectors)ASUS_I006D:/ # ASUS_I006D:/ # lpdump -s 1 | grep "^super:"super: 2048 .. 2897112: system_b (2895064 sectors)super: 2897920 .. 3504840: system_ext_b (606920 sectors)super: 3505152 .. 3507896: odm_b (2744 sectors)super: 3508224 .. 4272624: product_b (764400 sectors)super: 4273152 .. 6884696: vendor_b (2611544 sectors)super: 9566208 .. 9668608: new_part001 (102400 sectors)ASUS_I006D:/ # ASUS_I006D:/ # mkdir -p /data/local/tmp/aASUS_I006D:/ # ASUS_I006D:/ # chmod 1777 /data/local/tmp/aASUS_I006D:/ # ASUS_I006D:/ # /data/adb/magisk/busybox chcon -R -v --reference=/data/local/tmp /data/local/tmp/acontext of /data/local/tmp/a retained as u:object_r:shell_data_file:s0ASUS_I006D:/ # ASUS_I006D:/ # mount /dev/block/mapper/new_part001 /data/local/tmp/aASUS_I006D:/ # ASUS_I006D:/ # df -h /data/local/tmp/a Filesystem      Size Used Avail Use% Mounted on/dev/block/dm-0 44M 32K   44M   1% /data/local/tmp/aASUS_I006D:/ # ASUS_I006D:/ # ls -l /data/local/tmp/atotal 28-rw-r--r-- 1 root root   262 2024-09-06 20:57 VERSIONdrwx------ 2 root root 16384 2024-09-06 20:11 lost+found-rw-r--r-- 1 root root     0 2024-09-06 20:11 workingASUS_I006D:/ # ASUS_I006D:/ # cat /data/local/tmp/a/VERSIONThis partition was created while Android OS asus/EU_I006D/ASUS_I006D:11/RKQ1.210303.002/33.1004.0610.60:user/release-keys was runningThe partition is still there after OS upgrade to asus/EU_I006D/ASUS_I006D:11/RKQ1.210303.002/33.1004.0610.60:user/release-keysASUS_I006D:/ # ASUS_I006D:/ #

-> The new dynamic partition with the data is still there

I also successfully tested the access to the new partition after installing LineageOS 20, LineageOS 21 and /e/.

To use the new dynamic partition with "normal" Android Apps, mount the partition to a mount point in /sdcard or one of the sub directories in that directory.

Example:

mkdir /sdcard/Documents/my_new_partition chown media_rw:media_rw /sdcard/Documents/my_new_partition # 777 is probably not really required here ... # chmod 1777 /sdcard/Documents/my_new_partition /data/adb/magisk/busybox chcon -R --reference=/storage/self/primary /sdcard/Documents/my_new_partition/mount /dev/block/mapper/new_part001 /sdcard/Documents/my_new_partition/

The new dynamic partition can also be accessed when the phone is booted from a TWRP image

Example:

ASUS_I006D:/ # getprop ro.system.build.fingerprintasus/twrp_I006D/I006D:99.87.36/SP2A.220405.004/xtrnaw711251716:eng/test-keysASUS_I006D:/ # ASUS_I006D:/ # mkdir -p /tmp/aASUS_I006D:/ # ASUS_I006D:/ # mount /dev/block/mapper/new_part001 /tmp/aASUS_I006D:/ # ASUS_I006D:/ # df -h /tmp/aFilesystem      Size Used Avail Use% Mounted on/dev/block/dm-0 44M 32K   44M   1% /tmp/aASUS_I006D:/ # ASUS_I006D:/ # ls -l /tmp/atotal 32-rw-r--r-- 1 root root   262 2024-09-06 13:57 VERSIONdrwx------ 2 root root 16384 2024-09-06 13:11 lost+found-rw-r--r-- 1 root root     0 2024-09-06 13:11 workingASUS_I006D:/ #

The new dynamic partition can also be accessed when the phone is booted into the recovery of a LineageOS based ROM:

To access the new dynamic partition, enable adb in the GUI menue of the recovery and copy the binaries dmctl and lpdump to the phone .

Then execute the commands above to create the logical device for the dynamic partition:

Commands to access the new dynamic partition if the phone is booted from a LineageOS recovery

# get the data for the new dynamic partition new_part001 # (just ignore the error messages from lpdump) #ASUS_I006D:/tmp/system/bin # ./lpdump | grep new_part001lpdump E 02-24 01:42:34 792 792 images.cpp:64] [liblp]bool android::fs_mgr::IsEmptySuperImage(const std::string &) open failed: No such file or directorylpdump E 02-24 01:42:34 792 792 images.cpp:64] [liblp]bool android::fs_mgr::IsEmptySuperImage(const std::string &) open failed: No such file or directory Name: new_part001super: 9566208 .. 9668608: new_part001 (102400 sectors) ASUS_I006D:/tmp/system/bin # # create the logical device ASUS_I006D:/tmp/system/bin # ./dmctl create new_part001 linear 0 102400 /dev/block/by-name/super 9566208 ASUS_I006D:/tmp/system/bin # ASUS_I006D:/tmp/system/bin # ./dmctl list devices Available Device Mapper Devices:new_part001 : 253:0 ASUS_I006D:/tmp/system/bin # # ... and mount the logical device ASUS_I006D:/tmp/system/bin # mkdir /tmp/aASUS_I006D:/tmp/system/bin # ASUS_I006D:/tmp/system/bin # mount /dev/block/mapper/new_part001 /tmp/aASUS_I006D:/tmp/system/bin # ASUS_I006D:/tmp/system/bin # df -h /tmp/aFilesystem Size Used Avail Use% Mounted on/dev/block/dm-0 44M 32K 44M 1% /tmp/aASUS_I006D:/tmp/system/bin # ASUS_I006D:/tmp/system/bin # ls -l /tmp/a total 32-rw-r--r-- 1 root root 262 2024-09-06 18:57 VERSIONdrwx------ 2 root root 16384 2024-09-06 18:11 lost+found-rw-r--r-- 1 root root 0 2024-09-06 18:11 working ASUS_I006D:/tmp/system/bin #

Note that this approach to create new dynamic partitions also works for more then one new dynamic partition as long as there is enough free space in the super partition for the new dynamic partitions.

Resume

New dynamic partitions are the perfect storage location for important data that should not be lost if something really bad happens and a factory reset is required.

Remember, however, that you will have to delete these partitions manually if you want to clean up the data on the phone (e.g. before selling it).
A factory reset or a re-installation of the OS via TWRP or sideload does NOT delete the new partitions.
Even the installation via fastboot mode (see How to install "fastboot images" for details) does not delete the new partition.

Note that the data stored in these partitions is not encrypted by the operating system; if encryption is required, it must be implemented using other tools.

And, as long as you do not increase the size for the partition super, the available space for the new dynamic partitions is limited

Test Environments

Hardware

All tests were done on an ASUS Zenfone 8

OS

Original Android 13 from ASUS for the ASUS Zenfone 8
OmniROM 14 (= Android 14)
LineageOS 20 (= Android 13)
LineageOS 21 (= Android 14)
/e/ 2.0 = (Android 13)

Trouble Shooting

If you reduce the free space in the partition super too much, the next operating system update may fail with an error message like this:

update_engine_sideload E 09-06 10:12:58 654 654 [ERROR:dynamic_partition_control_android.cc(1032)] Cannot resize partition vendor_b to size 1334673408. Not enough space?

The error message on the console is then something like

Error applying update: 7 (ErrorCode: kIinstallDeviceOpenError)

To fix it, boot the phone into the fasboot mode and delete one or more of your new dynamic partitions.

Creating a new filesystem with ext4 or ext3 fails with an error like this:

ASUS_I006D:/ # mkfs.ext4 /dev/block/dm-10mke2fs 1.46.6 (1-Feb-2023)Invalid filesystem option set: has_journal,extent,huge_file,flex_bg,metadata_csum,metadata_csum_seed,64bit,dir_nlink,extra_isize,orphan_file1|ASUS_I006D:/ #

The reason for this error are the default values for new filesystems in the file /system/etc/mke2fs.conf - either change them, or use another filesystem type.

If access to the files in the new partition does not work for an Android app, check wether the access works with SELinux disabled.

To temporary disable SELinux execute as user root in an (adb) session:

setenforce 0

If the access works with SELinux disabled, one or more SELinux rules are missing. Either add the missing SELinux rules (using for example the Magisk executable magiskpolicy; see also How to add missing SELinux policies dynamically using Magisk) or temporary disable SELinux while using the files in the new partition.

There is a small app called "SELinuxModeChanger" on F-Droid to change the SELinux status:

https://f-droid.org/repo/com.mrbimc.selinux_20171031.apk

That app also works on Android 14 and can be used temporary disable SELinux via GUI on the phone.

Notes

The binary dmctl is part of the Android OS; in most Android OS versions the binary lpdump is also part of the OS.

lpadd (and also lpdump) are part of the RO2RW tools that can be downloaded from here: https://sourceforge.net/projects/multi-function-patch/
Both tools are also included in the Magisk Module ro2rw_3.7.3.0.zip (see also the list of available Magisk Modules on this website below)

The script to list the logical device usage in the partition super is available here: list_logical_device_usage.sh

see How to create logical devices using the device mapper from Android for more details about the usage of dynamic partitions in Android

see also How to create logical devices for disk image files

For the ASUS Zenfone 8 you can get rid of the new dynamic partitions installing one of the raw Android images for that phone (see https://xdaforums.com/t/full-recover-to-stock-if-things-went-really-bad.4337467/). But I don't know if this kind of raw images are available for other phones also.

History of this entry

07.09.2024

initial release

How to access the system partitions while the phone is booted into the LineageOS recovery

URL: https://xdaforums.com/t/guide-how-to-access-the-system-partition-while-the-phone-is-booted-into-the-lineageos-recovery.4691374/

How to access the system partition while the phone is booted into the LineageOS recovery

Unfortunately the recovery from the LineageOS is very limited and not really useful to fix errors. But in the end it's still a small Linux system with root access.

In the LineageOS recovery no access to the Android system partitions is possible; except that you can mount the partition for the root filesystem / (this is the dynamic partition system_a or system_b depending on the current active slot) via recovery GUI ( the menu entry is Advancded / Mount/umount system).

If access to the other system partitions from within the LineageOS recovery is necessary the following approach can be used:

First enable the access via adb in the recovery GUI (-> The menu entry is "Advanced / Enable ADB"; note that the menu entry is not visible anymore after enabling adb)

Next copy the necessary binaries to access the system partitions to the phone:

These are the binaries lpdump and dmctrl.

The easiest way to access the system partitions is to use my script remount_dynamic_partitions.sh .

To use that script awk must also be available on the phone. In most of the LIneageOS recoveries awk is not available - therefor awk must be copied to the phone before using the script.

So, to use the script copy the binaries lpdump, dmctrl, and awk and the script remount_dynamic_partitions.sh to the directory /tmp on the phone via adb and issue the following commands in an adb shell;

chmod 755 /tmp/remount_dynamic_partitions.shchmod 755 /tmp/dmctlchmod 755 /tmp/lpdumpchmod 755 /tmp/awkexport PATH=$PATH:/tmp# this must be in one line! #TMPDIR=/tmp BASEDIR=/tmp DELETE_SCRIPT_NAME=/tmp/delete_dm_devices.sh UMOUNT_SCRIPT_NAME=/tmp/umount_dm_devices.sh /tmp/remount_dynamic_partitions.sh

Example output

ASUS_I006D:/tmp # TMPDIR=/tmp BASEDIR=/tmp DELETE_SCRIPT_NAME=/tmp/delete_dm_devices.sh UMOUNT_SCRIPT_NAME=/tmp/umount_dm_devices.sh ./remount_dynamic_partitions.sh remount_dynamic_partitions.sh 1.1.0 - create new additional logical devices for the dynamic partitions in the Android super partition and mount themRetrieving the list of logical partitions for the slot "_a" in the super partition ...The logical partitions for the slot "_a" in the super partition are: new_part001 odm_a product_a system_a system_ext_a vendor_a vendor_dlkm_a Mounting the volumes in the super partition for the slot "_a" to sub directories in the directory "/tmp/a" ... The mount options used are "-o ro" # Processing the logical partition "new_part001" ...The mount point for the logical partition "new_part001" is "/tmp/a/new_part001"Creating the DM mapper device "dm-a-new_part001" for the logical partition "new_part001" ... # Processing the logical partition "odm_a" ...The mount point for the logical partition "odm_a" is "/tmp/a/odm_a"Creating the DM mapper device "dm-a-odm_a" for the logical partition "odm_a" ... # Processing the logical partition "product_a" ...The mount point for the logical partition "product_a" is "/tmp/a/product_a"Creating the DM mapper device "dm-a-product_a" for the logical partition "product_a" ... # Processing the logical partition "system_a" ...The mount point for the logical partition "system_a" is "/tmp/a/system_a"Creating the DM mapper device "dm-a-system_a" for the logical partition "system_a" ... # Processing the logical partition "system_ext_a" ...The mount point for the logical partition "system_ext_a" is "/tmp/a/system_ext_a"Creating the DM mapper device "dm-a-system_ext_a" for the logical partition "system_ext_a" ... # Processing the logical partition "vendor_a" ...The mount point for the logical partition "vendor_a" is "/tmp/a/vendor_a"Creating the DM mapper device "dm-a-vendor_a" for the logical partition "vendor_a" ... # Processing the logical partition "vendor_dlkm_a" ...The mount point for the logical partition "vendor_dlkm_a" is "/tmp/a/vendor_dlkm_a"Creating the DM mapper device "dm-a-vendor_dlkm_a" for the logical partition "vendor_dlkm_a" ... Creating the script to umount the mount points for the new devices "/tmp/umount_dm_devices.sh" ...-rwxr-xr-x 1 root root 655 1970-02-24 02:49 /tmp/umount_dm_devices.sh Creating the script to delete the temporary devices "/tmp/delete_dm_devices.sh" ...-rwxr-xr-x 1 root root 1334 1970-02-24 02:49 /tmp/delete_dm_devices.sh Use /tmp/umount_dm_devices.sh yes ; /tmp/delete_dm_devices.sh yes to umount and delete the temporary devices Mounted logical partitions are: /dev/block/dm-7 44M 32K 44M 1% /tmp/a/new_part001/dev/block/dm-8 1.3M 1.3M 8.0K 100% /tmp/a/odm_a/dev/block/dm-9 590M 562M 28M 96% /tmp/a/product_a/dev/block/dm-10 1.3G 1.2G 26M 99% /tmp/a/system_a/dev/block/dm-11 336M 308M 29M 92% /tmp/a/system_ext_a/dev/block/dm-12 977M 947M 29M 97% /tmp/a/vendor_a/dev/block/dm-13 34M 34M 104K 100% /tmp/a/vendor_dlkm_a

The result of the script is:

ASUS_I006D:/tmp # ASUS_I006D:/tmp # mount | grep /tmp/a/dev/block/dm-7 on /tmp/a/new_part001 type ext3 (ro,seclabel,relatime)/dev/block/dm-8 on /tmp/a/odm_a type ext4 (ro,seclabel,relatime)/dev/block/dm-9 on /tmp/a/product_a type ext4 (ro,seclabel,relatime)/dev/block/dm-10 on /tmp/a/system_a type ext4 (ro,seclabel,relatime)/dev/block/dm-11 on /tmp/a/system_ext_a type ext4 (ro,seclabel,relatime)/dev/block/dm-12 on /tmp/a/vendor_a type ext4 (ro,seclabel,relatime)/dev/block/dm-13 on /tmp/a/vendor_dlkm_a type ext4 (ro,seclabel,relatime) ASUS_I006D:/tmp #

In some LineageOS based OS, the partitions can be mounted in read/write mode while booted into the recovery. To mount the system partitions in read/write mode, use this command:

MOUNT_OPTIONS="-o rw" TMPDIR=/tmp BASEDIR=/tmp DELETE_SCRIPT_NAME=/tmp/delete_dm_devices.sh UMOUNT_SCRIPT_NAME=/tmp/umount_dm_devices.sh /tmp/remount_dynamic_partitions.sh

Notes

see How to create logical devices using the device mapper from Android for details about the usage of dynamic partitions and logical devices in Android; the section also contains the necessary infos to enable the access manually using the lpdump and dmctl commands

The dmctl binary is part of the LineageOS; the lpdump binary from the LineageOS needs additional libraries, therefor I recommend to use the lpdump binary from the RO2RW tools available here: https://sourceforge.net/projects/multi-function-patch/

Or use the lpdump binary from the Magisk Module ro2rw_3.7.3.0.zip:

To extract only the binary lpdump for arm64 CPUs rom the Magisk Module use this command:

unzip -p ro2rw_3.7.3.0.zip ro2rw_executables/arm64/lpdump >lpdump

For awk use either the awk binary from the LineageOS or download the awk binary for arm64 CPUs from here: awk.

History of this entry

07.09.2024

initial release

How to create logical devices for disk image files

URL: https://xdaforums.com/t/guide-how-to-create-logical-devices-for-disk-image-files.4691642/

How to create logical devices for disk image files

Sometimes it is useful to have a separate disk for the data of an application. Unfortunately, it doesn't really make sense to connect external hard disks to a phone. But Android is Linux and therefore virtual hard disks can be created just like in any other Linux operating system.

Example:

Note: The commands must be done as user root.

Create a file that is used as virtual disk; in this example we create a small disk with only 100 MB:

ASUS_I006D:/data/local/tmp # dd if=/dev/zero of=./virtual_disk_100mb bs=100m count=11+0 records in1+0 records out104857600 bytes (100 M) copied, 0.070 s, 1.3 G/sASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # du -hs virtual_disk_100mb100M    virtual_disk_100mbASUS_I006D:/data/local/tmp #

Next create an LOFI device:

ASUS_I006D:/data/local/tmp # LOFIDEV=$( losetup -f )ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # ls -l $LOFIDEV brw------- 1 root root 7, 296 2024-09-08 12:53 /dev/block/loop37 ASUS_I006D:/data/local/tmp #

Attach the file just created to the LOFI device:

ASUS_I006D:/data/local/tmp # losetup $LOFIDEV /data/local/tmp/virtual_disk_100mbASUS_I006D:/data/local/tmp #
Create a logical device using dmctl for the LOFI device:

130|ASUS_I006D:/data/local/tmp # blockdev --getsize $LOFIDEV 204800 ASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # dmctl create virtualdisk001 linear 0 204800 $LOFIDEV 0ASUS_I006D:/data/local/tmp #

Create a filesystem on the new logical device:

ASUS_I006D:/data/local/tmp # mkfs.ext3 /dev/block/mapper/virtualdisk001mke2fs 1.46.6 (1-Feb-2023)128-byte inodes cannot handle dates beyond 2038 and are deprecatedDiscarding device blocks: done                           Creating filesystem with 102400 1k blocks and 25688 inodesFilesystem UUID: c30b6de5-2290-4421-819b-06f9fa7334f6Superblock backups stored on blocks:     8193, 24577, 40961, 57345, 73729Allocating group tables: done                           Writing inode tables: done                           Creating journal (4096 blocks): doneWriting superblocks and filesystem accounting information: doneASUS_I006D:/data/local/tmp #

and finally mount the new logical device:

ASUS_I006D:/data/local/tmp # mkdir -p /data/local/tmp/aASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # mount /dev/block/mapper/virtualdisk001 /data/local/tmp/aASUS_I006D:/data/local/tmp # ASUS_I006D:/data/local/tmp # df -h /data/local/tmp/aFilesystem      Size Used Avail Use% Mounted on/dev/block/dm-8 93M 30K   93M   1% /data/local/tmp/aASUS_I006D:/data/local/tmp #

Note

To enable access by non-root user to the new filesystem, change the standard permissions as required and set the SELinux context, example:

/data/adb/magisk/busybox chcon -R -h -v --reference=/data/local/tmp /data/local/tmp/a/

For general access to the new filesystem it should be mounted to a directory in /sdcard/ or a sub directory.

To automatically mount the virtual disk, a start script for Magisk (or a similar tool to implement init scripts for Android) is required; the script might look like this:

Example:

cat /data/adb/service.d/mount_virtual_disk0.sh

ASUS_I006D:/ # cat /data/adb/service.d/mount_virtual_disk0.sh# for debugging# exec 1>/data/cache/${0##*/}.log 2>&1 set -xVIRTUAL_DISK_FILE="/data/local/tmp/virtual_disk_100mb"VIRTUAL_DISK_MOUNT_POINT="/data/local/tmp/virtual_disk_001"VIRTUAL_DISK_LOGICAL_DEVICE="/dev/block/mapper/virtualdisk001"if [ -r "${VIRTUAL_DISK_FILE}" ] ; then LOFIDEV=$( losetup -f ) if [ "${LOFIDEV}"x != ""x ] ; then losetup "${LOFIDEV}" "${VIRTUAL_DISK_FILE}" LOFI_DEV_SIZE=$( blockdev --getsize "${LOFIDEV}" ) dmctl create "${VIRTUAL_DISK_LOGICAL_DEVICE##*/}" linear 0 ${LOFI_DEV_SIZE} ${LOFIDEV} 0 if [ $? -eq 0 ] ; then mkdir -p "${VIRTUAL_DISK_MOUNT_POINT}" if [ -d "${VIRTUAL_DISK_MOUNT_POINT}" ] ; then mount "${VIRTUAL_DISK_LOGICAL_DEVICE}" "${VIRTUAL_DISK_MOUNT_POINT}" fi else echo "ERROR: Can not create a logical device using dmctrl" fi else echo "ERROR: Can not create a LOFI device" fifiASUS_I006D:/ #

This approach can also be used to mount an existing image file, e.g. to mount vendor.img for a Android ROM copy the file to the phone and execute these commands:

Create an logical device for an existing image file

ASUS_I006D:/ # ls -ltr /sdcard/Download/vendor.img-rw-rw---- 1 u0_a233 media_rw 1337110528 2024-09-07 12:24 /sdcard/Download/vendor.img ASUS_I006D:/ # ASUS_I006D:/ # LOFIDEV=$( losetup -f ) ASUS_I006D:/ # ASUS_I006D:/ # ls -l $LOFIDEVbrw------- 1 root root 7, 312 2024-09-09 08:33 /dev/block/loop39 ASUS_I006D:/ # ASUS_I006D:/ # losetup $LOFIDEV /sdcard/Download/vendor.img ASUS_I006D:/ # ASUS_I006D:/ # mkdir -p /sdcard/vendor ASUS_I006D:/ # ASUS_I006D:/ # blockdev --getsize $LOFIDEV 2611544 ASUS_I006D:/ # ASUS_I006D:/ # dmctl create vendor_img linear 0 2611544 $LOFIDEV 0 ASUS_I006D:/ # ASUS_I006D:/ # mount /dev/block/mapper/vendor_img /sdcard/vendor ASUS_I006D:/ # ASUS_I006D:/ # df -h /sdcard/vendor/Filesystem Size Used Avail Use% Mounted on/dev/block/dm-8 1.2G 1.2G 3.8M 100% /mnt/user/0/emulated/0/vendor ASUS_I006D:/ # ASUS_I006D:/ # ls -l /sdcard/vendor/ total 180drwxr-xr-x 2 root shell 4096 2009-01-01 01:00 apexdrwxr-xr-x 7 root shell 4096 2009-01-01 01:00 appdrwxr-xr-x 2 root shell 4096 2009-01-01 01:00 asusfwdrwxr-x--x 3 root shell 12288 2009-01-01 01:00 bindrwxr-xr-x 2 root shell 4096 2009-01-01 01:00 bt_firmware-rw------- 1 root root 13500 2009-01-01 01:00 build.propdrwxr-xr-x 2 root shell 4096 2009-01-01 01:00 dspdrwxr-xr-x 31 root shell 8192 2009-01-01 01:00 etclrw-r--r-- 1 root root 19 2009-01-01 01:00 factory -> /mnt/vendor/persistdrwxr-xr-x 5 root shell 8192 2009-01-01 01:00 firmwaredrwxr-xr-x 2 root shell 4096 2009-01-01 01:00 firmware_mntdrwxr-xr-x 10 root shell 32768 2009-01-01 01:00 libdrwxr-xr-x 10 root shell 36864 2009-01-01 01:00 lib64drwx------ 2 root root 16384 2009-01-01 01:00 lost+foundlrw-r--r-- 1 root root 4 2009-01-01 01:00 odm -> /odmdrwxr-xr-x 3 root shell 4096 2009-01-01 01:00 odm_dlkmdrwxr-xr-x 2 root shell 4096 2009-01-01 01:00 overlaydrwxr-xr-x 5 root shell 4096 2009-01-01 01:00 rfsdrwxr-xr-x 3 root shell 4096 2009-01-01 01:00 vendor_dlkmdrwxr-xr-x 2 root shell 4096 2009-01-01 01:00 xrom ASUS_I006D:/ #

---

Trouble Shooting

Error message like this

ASUS_I006D:/ # mount -o ro /dev/block/mapper/vendor_img /sdcard/vendormount: /dev/block/mapper/vendor_img: need -t1|ASUS_I006D:/
are most likely due to the fact that either no known file system is used for the image or the size of the logical device created with dmctl does not match the size of the loop device. The error message is also displayed if the image cannot be mounted in read/write mode.

Notes

see also How to create a new dynamic partition in the partition super

History of this entry

08.09.2024

initial release

How to compile the OrangeFox Recovery for the ASUS Zenfone 8

URL: https://xdaforums.com/t/how-to-compile-the-orangefox-recovery-for-the-asus-zenfone-8.4692137/

How to compile the OrangeFox Recovery for the ASUS Zenfone 8

The OrangeFox Recovery (https://wiki.orangefox.tech/) is a recovery for Android phones based on TWRP. There is currently no official build of the OrangeFox recovery image for the ASUS Zenfone 8 (see https://orangefox.download/ for the list of officially supported phones) but it's quite simple to create an OrangeFox recovery image for the ASUS Zenfone 8.

There is a detailed guide on how to create an OrangeFox recovery image: https://wiki.orangefox.tech/en/dev/building. This guide is sufficient to create an OrangeFox recovery image for someone who knows how to build an Android ROM.

In this post, I document the steps I took to successfully compile an OrangeFox recovery image for the ASUS Zenfone 8 using the instructions from that page.

First create and init the repository for the OrangeFox recovery image using the instructions from the OrangeFox wiki:

cd /devpool001/develop/orangefox/mkdir ./OrangeFox_synccd ./OrangeFox_syncgit clone https://gitlab.com/OrangeFox/sync.gitcd ./sync/./orangefox_sync.sh --branch 12.1 --path /devpool001/develop/orangefox/fox_12.1

The repositories for the OrangeFox recovery images are now in the directory /devpool001/develop/orangefox/fox_12.1.

Now we need the device tree for the ASUS Zenfone 8. Because there is already a TRWP image for the ASUS Zenfone 8 it's not necessary to create a device tree for that phone from scratch. The lunch command automatically downloads the device tree from the TWRP repositories for the phone.

Therefor to start the build use these commands:

cd /devpool001/develop/orangefox/fox_12.1export ALLOW_MISSING_DEPENDENCIES=trueexport FOX_BUILD_DEVICE=I006Dexport LC_ALL="C"# This variable is necessary for installing the recovery via zip file#export TARGET_DEVICE_ALT="gssi,ASUS_I006D"source build/envsetup.shlunch twrp_I006D-engmka adbd bootimage

Example output

[ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ ls -l device/ total 1 drwxrwxr-x. 4 xtrnaw7 xtrnaw7 4 Sep 10 22:15 qcom drwxrwxr-x. 2 xtrnaw7 xtrnaw7 2 Sep 10 22:19 xiaomi [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ export ALLOW_MISSING_DEPENDENCIES=true [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ export FOX_BUILD_DEVICE=I006D [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ export LC_ALL="C" [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ export TARGET_DEVICE_ALT="gssi,ASUS_I006D" [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ source build/envsetup.sh [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ lunch twrp_I006D-eng In file included from build/make/core/config.mk:313: In file included from build/make/core/envsetup.mk:312: build/make/core/product_config.mk:160: error: Can not locate config makefile for product "twrp_I006D". 13:00:32 dumpvars failed with: exit status 1 Device I006D not found. Attempting to retrieve device repository from TeamWin Github (http://github.com/TeamWin). Found repository: android_device_asus_I006D Checking branch info Calculated revision: android-12.1 Adding dependency: Repository: android_device_asus_I006D Branch: android-12.1 Remote: TeamWin Path: device/asus/I006D Syncing repository to retrieve project. Fetching: 100% (1/1), done in 5.643s repo sync has finished successfully. Repository synced! Looking for dependencies Adding dependencies to manifest Adding dependency: Repository: android_device_asus_sm8350-common Branch: android-12.1 Remote: TeamWin Path: device/asus/sm8350-common Syncing dependencies Fetching: 100% (1/1), done in 2.980s repo sync has finished successfully. Looking for dependencies device/asus/sm8350-common has no additional dependencies. Done ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=99.87.36 TARGET_PRODUCT=twrp_I006D TARGET_BUILD_VARIANT=eng TARGET_BUILD_TYPE=release TARGET_ARCH=arm64 TARGET_ARCH_VARIANT=armv8-a TARGET_CPU_VARIANT=generic TARGET_2ND_ARCH=arm TARGET_2ND_ARCH_VARIANT=armv8-a TARGET_2ND_CPU_VARIANT=generic HOST_ARCH=x86_64 HOST_2ND_ARCH=x86 HOST_OS=linux HOST_OS_EXTRA=Linux-6.10.8-100.fc39.x86_64-x86_64-Fedora-Linux-39-(MATE-Compiz) HOST_CROSS_OS=windows HOST_CROSS_ARCH=x86 HOST_CROSS_2ND_ARCH=x86_64 HOST_BUILD_TYPE=release BUILD_ID=SP2A.220405.004 OUT_DIR=/devpool001/develop/orangefox/fox_12.1/out PRODUCT_SOONG_NAMESPACES=device/asus/sm8350-common ============================================ [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ du -hs device/asus/* 20M device/asus/I006D 19M device/asus/sm8350-common [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ [ OrangeFox DEV - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ mka adbd bootimagebuild/make/core/soong_config.mk:194: warning: BOARD_PLAT_PUBLIC_SEPOLICY_DIR has been deprecated. Use SYSTEM_EXT_PUBLIC_SEPOLICY_DIRS instead. build/make/core/soong_config.mk:195: warning: BOARD_PLAT_PRIVATE_SEPOLICY_DIR has been deprecated. Use SYSTEM_EXT_PRIVATE_SEPOLICY_DIRS instead. ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=99.87.36 TARGET_PRODUCT=twrp_I006D TARGET_BUILD_VARIANT=eng TARGET_BUILD_TYPE=release TARGET_ARCH=arm64 TARGET_ARCH_VARIANT=armv8-a TARGET_CPU_VARIANT=generic TARGET_2ND_ARCH=arm TARGET_2ND_ARCH_VARIANT=armv8-a TARGET_2ND_CPU_VARIANT=generic HOST_ARCH=x86_64 HOST_2ND_ARCH=x86 HOST_OS=linux HOST_OS_EXTRA=Linux-6.10.8-100.fc39.x86_64-x86_64-Fedora-Linux-39-(MATE-Compiz) HOST_CROSS_OS=windows HOST_CROSS_ARCH=x86 HOST_CROSS_2ND_ARCH=x86_64 HOST_BUILD_TYPE=release BUILD_ID=SP2A.220405.004 OUT_DIR=/devpool001/develop/orangefox/fox_12.1/out PRODUCT_SOONG_NAMESPACES=device/asus/sm8350-common....

-

When done, you should get a success message:

IMHO this approach should work for any phone that is officially supported by TWRP.

Trouble Shooting

Depending on the device tree used, building the image may fail with this error message:

[ 78% 246/313] including bootable/recovery/Android.mk ...FAILED: In file included from bionic/tests/libs/Android.build.dlext_testzip.mk:102:In file included from bootable/recovery/Android.mk:147:bootable/recovery/orangefox.mk:665: error: 'TW_MAX_BRIGHTNESS' is not set! You must provide a value for 'TW_MAX_BRIGHTNESS' in your device tree.07:13:23 ckati failed with: exit status 1#### failed to build some targets (03:16 (mm:ss)) ####

To fix it, add the missing variable TW_MAX_BRIGHTNESS to the file BoardConfig.mk:

[xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1]$ diff device/asus/I006D/BoardConfig.mk device/asus/I006D/BoardConfig.mk.org36c36< TW_MAX_BRIGHTNESS := 255---> [xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1]$

and restart the build command:

mka adbd bootimage

If the lunch command does not find the repositories with the device tree for the phone, add the repositories for the device tree manually, example:

To add the repositories with the device tree for the ASUS Zenfone 8 create the file ./.repo/local_manifests/roomservice.xml with the necessary repositories:

[ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $ cat .repo/local_manifests/roomservice.xml <?xml version="1.0" encoding="UTF-8"?> <manifest> <project path="device/asus/I006D" remote="TeamWin" name="android_device_asus_I006D" revision="android-12.1" /> <project path="device/asus/sm8350-common" remote="TeamWin" name="android_device_asus_sm8350-common" revision="android-12.1" /> </manifest> [ OmniRom 14 Dev - xtrnaw7@t15g /devpool001/develop/orangefox/fox_12.1 ] $and start another sync:

repo sync

The repo sync command prints some error messages that can be ignored as far as I know. The result after the repo sync should look like this:
[ OmniRom 14 Dev - xtrnaw7@t15g /datapool001/develop/orangefox/fox_12.1 ] $ du -hs device/asus/*20M device/asus/I006D20M device/asus/sm8350-common[ OmniRom 14 Dev - xtrnaw7@t15g /datapool001/develop/orangefox/fox_12.1 ] $

Now use the commands listed above to start the build process

If the installation of the ZIP file with the OrangeFox recovery fails with an error message about a mismatched product ID between the phone and the image in the ZIP file, add the product ID to the environment variable TARGET_DEVICE_ALT and create a new zip file.

Notes

see also How to compile TWRP for the ASUS Zenfone 8

see also these OrangeFox recovery related Webpages:

    OrangeFox Recovery Home page: https://wiki.orangefox.tech/

    Build instructions for the OrangeFox recovery: https://wiki.orangefox.tech/en/dev/building

    Instruction to sync the OrangeFox recovery repos: https://gitlab.com/OrangeFox/sync

    List fo custom build variables for creating the OrangeFox recovery image: https://gitlab.com/OrangeFox/vendor/recovery/-/blob/master/orangefox_build_vars.txt

The initial repository for the OrangeFox recovery image requires about 22 GB disk space; after a build of the OrangeFox recovery image the repository requires about 28 GB disk space.

The installation the ZIP file created for the OrangeFox recovery works after booting from TWRP or from the OrangeFox recovery image.

Booting the phone from the image file with the OrangeFox recovery works; installing the OrangeFox recovery via the recovery GUI entry "Flash current OrangeFox" works also. This installation method does not install the OrangeFox addons.

To copy the OrangeFox addons manually to the phone, copy the directory ./vendor/recovery/FoxFiles/ from the OrangeFox repository to the directory /sdcard/Fox on the phone:
[xtrnaw7@t15g /devpool001/develop/orangefox_test/fox_12.1]$ ls -l ./vendor/recovery/FoxFiles/ total 21921drwxrwxr-x. 2 xtrnaw7 xtrnaw7        3 Sep 10 22:20 AromaFM-rwxrwxr-x. 1 xtrnaw7 xtrnaw7 11278270 Sep 10 22:20 Magisk.zip-rwxrwxr-x. 1 xtrnaw7 xtrnaw7 11278270 Sep 10 22:20 Magisk_uninstall.zip-rw-rw-r--. 1 xtrnaw7 xtrnaw7 1865207 Sep 10 22:20 OF_initd.zip[xtrnaw7@t15g /devpool001/develop/orangefox_test/fox_12.1]$

The files are also in the OrangeFox recovery zip file :

qssi:/sdcard/Download # unzip -l OrangeFox-Unofficial-I006D.zip | grep sdcard/                                                                                                                        11278270 2008-02-29 03:33   sdcard/Fox/FoxFiles/Magisk_uninstall.zip 1756712 2008-02-29 03:33   sdcard/Fox/FoxFiles/AromaFM/AromaFM.zip 11278270 2008-02-29 03:33   sdcard/Fox/FoxFiles/Magisk.zip      911 2008-02-29 03:33   sdcard/Fox/fox.bashrcqssi:/sdcard/Download #

An in-official OrangeFox recovery image for the ASUS Zenfone 8 is available for download on my webpage - see below.

History of this entry

11.09.2024

initial release

How to install Magisk modules via a script

URL: https://xdaforums.com/t/guide-how-to-install-magisk-modules-via-a-script.4694326/

How to install Magisk modules via a script

To speedup developing and testing Magisk Modules I wrote a shell script to install one or more Magisk Modules without using the Magisk GUI:

install_mm.sh

The usage for the script is:

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/Scripts ] $ s install_mm.sh v1.1.0 Usage: install_mm.sh [options_for_adb -- ] [--keep] [--force] [--reboot] [--dry-run] [magisk_module1|dir1 ... magisk_module#|dir#] Parameter: options_for_adb - options for the adb command; this parameter is only used if running on a PC --keep - do not delete the ZIP file with the Magisk Module (this parameter is only used if the script is running on a PC) --reboot - reboot the phone after installing all Magisk Modules --force - reboot the phone even if not all Magisk Modules could be installed --dry-run - only print the commands to install the Magisk modules Use the parameter "-h -v" to print the detailed usage help Environment variables used: TARGET_DIR = /sdcard/Download ADB_OPTIONS = TMPDIR = /tmp PREFIX = [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/Scripts ] $

Detailed usage help

[ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/Scripts ] $ ./install_mm.sh -h -v install_mm.sh v1.1.0 Usage: install_mm.sh [options_for_adb -- ] [--keep] [--force] [--reboot] [--dry-run] [magisk_module1|dir1 ... magisk_module#|dir#] Parameter: options_for_adb - options for the adb command; this parameter is only used if running on a PC --keep - do not delete the ZIP file with the Magisk Module (this parameter is only used if the script is running on a PC) --reboot - reboot the phone after installing all Magisk Modules --force - reboot the phone even if not all Magisk Modules could be installed --dry-run - only print the commands to install the Magisk module Use the parameter "-h -v" to print the detailed usage help “options_for_adb” are the options for the ‘adb’ command. If the parameter for the adb options is missing, the script uses the adb options stored in the environment variable ADB_OPTIONS. If the environment variable ADB_OPTIONS is empty and there are no script parameters for adb options, adb is executed without options. The parameter “options_for_adb” overwrites the value of the environment variable ADB_OPTIONS Use the prefix “+” for “options_for_adb” to add the options to the adb options defined in the environment variable ADB_OPTIONS. The parameter “--” is mandatory if adb options are specified in the parameter. The options for adb are optional; the script does not check the options for adb. The specification of “options_for_adb” is not permitted if the script is executed in a shell on the phone. "magisk_module#" is the name of a zip file with a Magisk Module to install; "dir#" is a directory tree with files with Magisk Mdodules. If "dir#" is used, the script installs all files with the extension .zip from that directory tree. The number of zip files or directories is only limited by the maxium parameter supported by the used shell. When running on the PC, the zip files are copied into the directory /sdcard/Download on the phone; to change that directory, set the environment variable TARGET_DIR before executing the script. The script deletes the zip file on the phone after successfully installing a Magisk Module when running on a PC; use the parameter "--keep" to keep the zip file on the phone. Set the environment variable PREFIX to the prefix for the commands to install the Magisk Modules. Example: The parameter "--dry-run" sets the variable PREFIX to "echo" to only print the commmands To use adb via WLAN enable adb via WiFi on the phone, start the adb for WLAN on the PC, example: adb -e -L tcp:localhost:5237 connect 192.168.1.148:6666 and then use a command like this to install Magisk Modules using the script: ./install_mm.sh -e -L tcp:localhost:5237 -- /data/Downloads/my_magisk_module.zip Environment variables used: TARGET_DIR = /sdcard/Download ADB_OPTIONS = TMPDIR = /tmp PREFIX = [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/Scripts ] $

The script uses the Magisk binary magisk with the parameter --install-module to install the Magisk Module.

The script can run either on a PC running Linux with a working adb connection to the phone or on the phone itself. The script uses the magisk binary to install the module. root access is required to install the module. The script uses "su -" to execute commands as user root if necessary.

Example for running the script on the PC

/data/develop/android/Scripts/install_mm.sh ./perl540_5.40.0.zip sqlite3_3.22.zip vim_9.1.672.zip

[xtrnaw7@t15g /data/backup/Android/MagiskModules]$ /data/develop/android/Scripts/install_mm.sh ./perl540_5.40.0.zip sqlite3_3.22.zip vim_9.1.672.zip install_mm.sh v1.1.0 started on Sat Sep 21 01:33:00 PM CEST 2024 Running on a PC Using adb to install the packages Using the prefix "su - -c " for root access The Magisk Modules are installed on the phone model ASUS_I006D with the serial number M6AIB760D0939LX Installing these Magisk Modules ./perl540_5.40.0.zip sqlite3_3.22.zip vim_9.1.672.zip *** Processing the Magisk Module "./perl540_5.40.0.zip" ... Installing the Magisk Module "./perl540_5.40.0.zip" ... Copying the file "./perl540_5.40.0.zip" to "/sdcard/Download/perl540_5.40.0.zip.541968" on the phone ... ./perl540_5.40.0.zip: 1 file pushed, 0 skipped. 36.2 MB/s (17373065 bytes in 0.458s) Installing the Magisk module file "/sdcard/Download/perl540_5.40.0.zip.541968" with Magisk ... - Current boot slot: _a - Device is system-as-root ************************** Perl 5.40 for arm64 by Bernd.Schemmer@gmx.de ************************** ******************* Powered by Magisk ******************* - Extracting module files The current environment for this installation is The version of the installed Magisk is "27.0" (27000) INFO: BOOTMODE is "true" INFO: MODPATH is "/data/adb/modules_update/perl540" INFO: TMPDIR is "/dev/tmp" INFO: ZIPFILE is "/storage/emulated/0/Download/perl540_5.40.0.zip.541968" INFO: ARCH is "arm64" INFO: IS64BIT is "true" INFO: API is "34" Installing the Magisk Module for Perl version "5.40.0" ... Checking the type of the CPU used in this device .... The CPU in this device is a arm64 CPU The machine type reported by "uname -m" is "aarch64" Renaming the existing directory "/data/local/tmp/perl540" to "/data/local/tmp/perl540.24626_2024_09-21" ... Copying the Perl files from "/data/adb/modules_update/perl540/perl540" to "/data/local/tmp/perl540" ... Correcting the permissions for the Perl files ... The Perl home directory "/data/local/tmp/perl_home" already exists Creating the script to set the environment for Perl "/system/bin/perl_env" ... Creating the wrapper script "/system/bin/perl_wrapper" ... Creating the linkw "/system/bin/perl" ... Creating the linkw "/system/bin/cpan" ... Creating the linkw "/system/bin/perlbug" ... Creating the linkw "/system/bin/perldoc" ... - Done "./perl540_5.40.0.zip" succcessfully installed Deleting the file "/sdcard/Download/perl540_5.40.0.zip.541968" on the phone ... *** Processing the Magisk Module "sqlite3_3.22.zip" ... Installing the Magisk Module "sqlite3_3.22.zip" ... Copying the file "sqlite3_3.22.zip" to "/sdcard/Download/sqlite3_3.22.zip.541968" on the phone ... sqlite3_3.22.zip: 1 file pushed, 0 skipped. 275.0 MB/s (1451571 bytes in 0.005s) Installing the Magisk module file "/sdcard/Download/sqlite3_3.22.zip.541968" with Magisk ... - Current boot slot: _a - Device is system-as-root ***************************************************************** SQLite3 Universal Binaries by adrianmmiller (Inspired by Olivier Pantale & aarch64 module) ***************************************************************** ******************* Powered by Magisk ******************* ************************************** * MMT Extended by Zackptg5 @ XDA * ************************************** - Extracting module files - Removing old files - Installing Checking for existing sqlite3 binary.... [Existing binary]: standard SQLite3 binary found in: /system/bin Install aborted ERROR: Error installing the Magisk Module "sqlite3_3.22.zip" *** Processing the Magisk Module "vim_9.1.672.zip" ... Installing the Magisk Module "vim_9.1.672.zip" ... Copying the file "vim_9.1.672.zip" to "/sdcard/Download/vim_9.1.672.zip.541968" on the phone ... vim_9.1.672.zip: 1 file pushed, 0 skipped. 36.6 MB/s (12532234 bytes in 0.327s) Installing the Magisk module file "/sdcard/Download/vim_9.1.672.zip.541968" with Magisk ... - Current boot slot: _a - Device is system-as-root ************************** vim for Android by Bernd.Schemmer@gmx.de ************************** ******************* Powered by Magisk ******************* - Extracting module files The current environment for this installation is The version of the installed Magisk is "27.0" (27000) INFO: BOOTMODE is "true" INFO: MODPATH is "/data/adb/modules_update/vim" INFO: TMPDIR is "/dev/tmp" INFO: ZIPFILE is "/storage/emulated/0/Download/vim_9.1.672.zip.541968" INFO: ARCH is "arm64" INFO: IS64BIT is "true" INFO: API is "34" Installing the Magisk Module with vim version "9.1.672" ... Checking the type of the CPU used in this device .... The CPU in this device is a arm64 CPU The machine type reported by "uname -m" is "aarch64" Correcting the permissions for the new binaries ... The vim data directory "/data/local/tmp/share/vim" already exists - Done "vim_9.1.672.zip" succcessfully installed Deleting the file "/sdcard/Download/vim_9.1.672.zip.541968" on the phone ... Installation summary ==================== 2 Magisk Module(s) successfully installed: ./perl540_5.40.0.zip vim_9.1.672.zip 1 Magisk Module(s) not installed: sqlite3_3.22.zip [xtrnaw7@t15g /data/backup/Android/MagiskModules]$

Example for running the script on the phone

/data/local/tmp/install_mm.sh selinux_tools_v1.4.zip ro2rw_3.7.3.0.zip

130|ASUS_I006D:/sdcard/Download $ /data/local/tmp/install_mm.sh selinux_tools_v1.4.zip ro2rw_3.7.3.0.zip install_mm.sh v1.1.0 started on Sat Sep 21 13:36:18 CEST 2024 Running on a phone Using the prefix "su - -c " for root access The Magisk Modules are installed on the phone model ASUS_I006D with the serial number M6AIB760D0939LX Installing these Magisk Modules selinux_tools_v1.4.zip ro2rw_3.7.3.0.zip *** Processing the Magisk Module "selinux_tools_v1.4.zip" ... Installing the Magisk Module "selinux_tools_v1.4.zip" ... Installing the Magisk module file "selinux_tools_v1.4.zip" with Magisk ... - Current boot slot: _a - Device is system-as-root ***************** SELinux_Tools by (see README) ***************** ******************* Powered by Magisk ******************* - Extracting module files The current environment for this installation is The version of the installed Magisk is "27.0" (27000) INFO: BOOTMODE is "true" INFO: MODPATH is "/data/adb/modules_update/selinux_tools" INFO: TMPDIR is "/dev/tmp" INFO: ZIPFILE is "/storage/emulated/0/Download/selinux_tools_v1.4.zip" INFO: ARCH is "arm64" INFO: IS64BIT is "true" INFO: API is "34" - Installing the Magisk Module with the SELinux tools "v1.4" ... Checking the type of the CPU used in this device .... The CPU in this device is a arm64 CPU The machine type reported by "uname -m" is "aarch64" Using the binaries from the directory "arm64-v8a" - Done "selinux_tools_v1.4.zip" succcessfully installed *** Processing the Magisk Module "ro2rw_3.7.3.0.zip" ... Installing the Magisk Module "ro2rw_3.7.3.0.zip" ... Installing the Magisk module file "ro2rw_3.7.3.0.zip" with Magisk ... - Current boot slot: _a - Device is system-as-root ************************** DynamicPartitionTools by see the module readme ************************** ******************* Powered by Magisk ******************* - Extracting module files The current environment for this installation is The version of the installed Magisk is "27.0" (27000) INFO: BOOTMODE is "true" INFO: MODPATH is "/data/adb/modules_update/ro2rw" INFO: TMPDIR is "/dev/tmp" INFO: ZIPFILE is "/storage/emulated/0/Download/ro2rw_3.7.3.0.zip" INFO: ARCH is "arm64" INFO: IS64BIT is "true" INFO: API is "34" Installing the Magisk Module with tools to process dynamic partitions version "3.7.3.0" ... Checking the type of the CPU used in this device .... The CPU in this device is a arm64 CPU The machine type reported by "uname -m" is "aarch64" Copying the files from "/data/adb/modules_update/ro2rw/ro2rw_executables/arm64" to "/data/adb/modules_update/ro2rw/system/bin/ro2rw" ... Correcting the permissions for the new binaries ... - Done "ro2rw_3.7.3.0.zip" succcessfully installed Installation summary ==================== 2 Magisk Module(s) successfully installed: selinux_tools_v1.4.zip ro2rw_3.7.3.0.zip ASUS_I006D:/sdcard/Download $

To only print the commands to install the Magisk Module without installing the module set the variable PREFIX to "echo" or something similar before executing the script, example:

PREFIX=echo /data/develop/android/Scripts/install_mm.sh /data/backup/Android/Test_EssentialMagiskModules/

[xtrnaw7@t15g /data/backup/Android/MagiskModules]$ PREFIX=echo /data/develop/android/Scripts/install_mm.sh /data/backup/Android/Test_EssentialMagiskModules/ install_mm.sh v1.1.0 started on Sat Sep 21 01:37:09 PM CEST 2024 Running on a PC Using adb to install the packages Using the prefix "su - -c " for root access Directory found in the parameter: Installing all zip files found in the directory "/data/backup/Android/Test_EssentialMagiskModules/" The Magisk Modules are installed on the phone model ASUS_I006D with the serial number M6AIB760D0939LX Installing these Magisk Modules /data/backup/Android/Test_EssentialMagiskModules/myscripts_1.1.0.0.zip /data/backup/Android/Test_EssentialMagiskModules/network_tools_1.0.0.zip /data/backup/Android/Test_EssentialMagiskModules/bootctl-binary-v2.1.3.zip /data/backup/Android/Test_EssentialMagiskModules/Nano_for_Android_NDK-6_3-6300.zip /data/backup/Android/Test_EssentialMagiskModules/DebugTools_9.1.2.0.zip /data/backup/Android/Test_EssentialMagiskModules/vim_9.1.672.zip WARNING: The value of the environment variable PREFIX is "echo" *** Processing the Magisk Module "/data/backup/Android/Test_EssentialMagiskModules/myscripts_1.1.0.0.zip" ... Installing the Magisk Module "/data/backup/Android/Test_EssentialMagiskModules/myscripts_1.1.0.0.zip" ... Copying the file "/data/backup/Android/Test_EssentialMagiskModules/myscripts_1.1.0.0.zip" to "/sdcard/Download/myscripts_1.1.0.0.zip.543119" on the phone ... /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/myscripts_1.1.0.0.zip /sdcard/Download/myscripts_1.1.0.0.zip.543119 Installing the Magisk module file "/sdcard/Download/myscripts_1.1.0.0.zip.543119" with Magisk ... /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/myscripts_1.1.0.0.zip.543119 "/data/backup/Android/Test_EssentialMagiskModules/myscripts_1.1.0.0.zip" succcessfully installed Deleting the file "/sdcard/Download/myscripts_1.1.0.0.zip.543119" on the phone ... /usr/bin/adb shell su - -c rm /sdcard/Download/myscripts_1.1.0.0.zip.543119 *** Processing the Magisk Module "/data/backup/Android/Test_EssentialMagiskModules/network_tools_1.0.0.zip" ... Installing the Magisk Module "/data/backup/Android/Test_EssentialMagiskModules/network_tools_1.0.0.zip" ... Copying the file "/data/backup/Android/Test_EssentialMagiskModules/network_tools_1.0.0.zip" to "/sdcard/Download/network_tools_1.0.0.zip.543119" on the phone ... /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/network_tools_1.0.0.zip /sdcard/Download/network_tools_1.0.0.zip.543119 Installing the Magisk module file "/sdcard/Download/network_tools_1.0.0.zip.543119" with Magisk ... /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/network_tools_1.0.0.zip.543119 "/data/backup/Android/Test_EssentialMagiskModules/network_tools_1.0.0.zip" succcessfully installed Deleting the file "/sdcard/Download/network_tools_1.0.0.zip.543119" on the phone ... /usr/bin/adb shell su - -c rm /sdcard/Download/network_tools_1.0.0.zip.543119 *** Processing the Magisk Module "/data/backup/Android/Test_EssentialMagiskModules/bootctl-binary-v2.1.3.zip" ... Installing the Magisk Module "/data/backup/Android/Test_EssentialMagiskModules/bootctl-binary-v2.1.3.zip" ... Copying the file "/data/backup/Android/Test_EssentialMagiskModules/bootctl-binary-v2.1.3.zip" to "/sdcard/Download/bootctl-binary-v2.1.3.zip.543119" on the phone ... /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/bootctl-binary-v2.1.3.zip /sdcard/Download/bootctl-binary-v2.1.3.zip.543119 Installing the Magisk module file "/sdcard/Download/bootctl-binary-v2.1.3.zip.543119" with Magisk ... /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/bootctl-binary-v2.1.3.zip.543119 "/data/backup/Android/Test_EssentialMagiskModules/bootctl-binary-v2.1.3.zip" succcessfully installed Deleting the file "/sdcard/Download/bootctl-binary-v2.1.3.zip.543119" on the phone ... /usr/bin/adb shell su - -c rm /sdcard/Download/bootctl-binary-v2.1.3.zip.543119 *** Processing the Magisk Module "/data/backup/Android/Test_EssentialMagiskModules/Nano_for_Android_NDK-6_3-6300.zip" ... Installing the Magisk Module "/data/backup/Android/Test_EssentialMagiskModules/Nano_for_Android_NDK-6_3-6300.zip" ... Copying the file "/data/backup/Android/Test_EssentialMagiskModules/Nano_for_Android_NDK-6_3-6300.zip" to "/sdcard/Download/Nano_for_Android_NDK-6_3-6300.zip.543119" on the phone ... /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/Nano_for_Android_NDK-6_3-6300.zip /sdcard/Download/Nano_for_Android_NDK-6_3-6300.zip.543119 Installing the Magisk module file "/sdcard/Download/Nano_for_Android_NDK-6_3-6300.zip.543119" with Magisk ... /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/Nano_for_Android_NDK-6_3-6300.zip.543119 "/data/backup/Android/Test_EssentialMagiskModules/Nano_for_Android_NDK-6_3-6300.zip" succcessfully installed Deleting the file "/sdcard/Download/Nano_for_Android_NDK-6_3-6300.zip.543119" on the phone ... /usr/bin/adb shell su - -c rm /sdcard/Download/Nano_for_Android_NDK-6_3-6300.zip.543119 *** Processing the Magisk Module "/data/backup/Android/Test_EssentialMagiskModules/DebugTools_9.1.2.0.zip" ... Installing the Magisk Module "/data/backup/Android/Test_EssentialMagiskModules/DebugTools_9.1.2.0.zip" ... Copying the file "/data/backup/Android/Test_EssentialMagiskModules/DebugTools_9.1.2.0.zip" to "/sdcard/Download/DebugTools_9.1.2.0.zip.543119" on the phone ... /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/DebugTools_9.1.2.0.zip /sdcard/Download/DebugTools_9.1.2.0.zip.543119 Installing the Magisk module file "/sdcard/Download/DebugTools_9.1.2.0.zip.543119" with Magisk ... /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/DebugTools_9.1.2.0.zip.543119 "/data/backup/Android/Test_EssentialMagiskModules/DebugTools_9.1.2.0.zip" succcessfully installed Deleting the file "/sdcard/Download/DebugTools_9.1.2.0.zip.543119" on the phone ... /usr/bin/adb shell su - -c rm /sdcard/Download/DebugTools_9.1.2.0.zip.543119 *** Processing the Magisk Module "/data/backup/Android/Test_EssentialMagiskModules/vim_9.1.672.zip" ... Installing the Magisk Module "/data/backup/Android/Test_EssentialMagiskModules/vim_9.1.672.zip" ... Copying the file "/data/backup/Android/Test_EssentialMagiskModules/vim_9.1.672.zip" to "/sdcard/Download/vim_9.1.672.zip.543119" on the phone ... /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/vim_9.1.672.zip /sdcard/Download/vim_9.1.672.zip.543119 Installing the Magisk module file "/sdcard/Download/vim_9.1.672.zip.543119" with Magisk ... /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/vim_9.1.672.zip.543119 "/data/backup/Android/Test_EssentialMagiskModules/vim_9.1.672.zip" succcessfully installed Deleting the file "/sdcard/Download/vim_9.1.672.zip.543119" on the phone ... /usr/bin/adb shell su - -c rm /sdcard/Download/vim_9.1.672.zip.543119 Installation summary ==================== 6 Magisk Module(s) successfully installed: /data/backup/Android/Test_EssentialMagiskModules/myscripts_1.1.0.0.zip /data/backup/Android/Test_EssentialMagiskModules/network_tools_1.0.0.zip /data/backup/Android/Test_EssentialMagiskModules/bootctl-binary-v2.1.3.zip /data/backup/Android/Test_EssentialMagiskModules/Nano_for_Android_NDK-6_3-6300.zip /data/backup/Android/Test_EssentialMagiskModules/DebugTools_9.1.2.0.zip /data/backup/Android/Test_EssentialMagiskModules/vim_9.1.672.zip [xtrnaw7@t15g /data/backup/Android/MagiskModules]$

Use grep to only print the commands that would be used to install the Magisk Modules, e.g:[xtrnaw7@t15g /data/backup/Android/MagiskModules]$ PREFIX=echo /data/develop/android/Scripts/install_mm.sh /data/backup/Android/Test_EssentialMagiskModules/ | grep "/adb" /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/myscripts_1.1.0.0.zip /sdcard/Download/myscripts_1.1.0.0.zip.553982 /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/myscripts_1.1.0.0.zip.553982 /usr/bin/adb shell su - -c rm /sdcard/Download/myscripts_1.1.0.0.zip.553982 /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/network_tools_1.0.0.zip /sdcard/Download/network_tools_1.0.0.zip.553982 /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/network_tools_1.0.0.zip.553982 /usr/bin/adb shell su - -c rm /sdcard/Download/network_tools_1.0.0.zip.553982 /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/bootctl-binary-v2.1.3.zip /sdcard/Download/bootctl-binary-v2.1.3.zip.553982 /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/bootctl-binary-v2.1.3.zip.553982 /usr/bin/adb shell su - -c rm /sdcard/Download/bootctl-binary-v2.1.3.zip.553982 /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/Nano_for_Android_NDK-6_3-6300.zip /sdcard/Download/Nano_for_Android_NDK-6_3-6300.zip.553982 /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/Nano_for_Android_NDK-6_3-6300.zip.553982 /usr/bin/adb shell su - -c rm /sdcard/Download/Nano_for_Android_NDK-6_3-6300.zip.553982 /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/DebugTools_9.1.2.0.zip /sdcard/Download/DebugTools_9.1.2.0.zip.553982 /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/DebugTools_9.1.2.0.zip.553982 /usr/bin/adb shell su - -c rm /sdcard/Download/DebugTools_9.1.2.0.zip.553982 /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/vim_9.1.672.zip /sdcard/Download/vim_9.1.672.zip.553982 /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/vim_9.1.672.zip.553982 /usr/bin/adb shell su - -c rm /sdcard/Download/vim_9.1.672.zip.553982 [xtrnaw7@t15g /data/backup/Android/MagiskModules]$
or use the parameter --dry-run:
[xtrnaw7@t15g /data/develop/android/Scripts]$ /data/develop/android/Scripts/install_mm.sh --dry-run --keep /data/backup/Android/Test_EssentialMagiskModules/ | grep "/adb" /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/myscripts_1.1.0.0.zip /sdcard/Download/myscripts_1.1.0.0.zip /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/myscripts_1.1.0.0.zip /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/network_tools_1.0.0.zip /sdcard/Download/network_tools_1.0.0.zip /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/network_tools_1.0.0.zip /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/bootctl-binary-v2.1.3.zip /sdcard/Download/bootctl-binary-v2.1.3.zip /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/bootctl-binary-v2.1.3.zip /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/Nano_for_Android_NDK-6_3-6300.zip /sdcard/Download/Nano_for_Android_NDK-6_3-6300.zip /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/Nano_for_Android_NDK-6_3-6300.zip /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/DebugTools_9.1.2.0.zip /sdcard/Download/DebugTools_9.1.2.0.zip /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/DebugTools_9.1.2.0.zip /usr/bin/adb push /data/backup/Android/Test_EssentialMagiskModules/vim_9.1.672.zip /sdcard/Download/vim_9.1.672.zip /usr/bin/adb shell su - -c magisk --install-module /sdcard/Download/vim_9.1.672.zip [xtrnaw7@t15g /data/develop/android/Scripts]$

The script is available for download here: install_mm.sh

Notes

see also Some hints for creating Magisk Modules and How to install packages (apk files) for Android via script

the Magisk Modules used in the examples are available here.

History of this entry

21.09.2024

initial release

Magisk Module documentation

Documentation for the Magisk Module with the network tools

URL: https://xdaforums.com/t/module-network-tools-for-cli-sessions.4696517/

Documentation for the Magisk Module with the network tools

The Magisk Module network_tools contains various network related CLI tools for Android:

Tool	Description	Homepage URL	Comment	added in version
bmon	bmon is a monitoring and debugging tool to capture networking related statistics and prepare them visually in a human friendly way	https://github.com/tgraf/bmon	bmon needs root access	1.0.0
ngrep	ngrep is a grep for network packets	https://github.com/jpr5/ngrep	ngrep needs root access	1.0.0
dig, mdig, host, nsloookup, nsupdate	dig is the Linux tool for hostname resolution via DNS server mdig, host, nslookup, and nsupdate can also be used for hostname resolution.	https://gitlab.isc.org/	The binaries dig, mdig, host, nslookup, and nsupdate are from bind 9.11 "mdig is a multiple/pipelined query version of dig: instead of waiting for a response after sending each query, it begins by sending all queries. Responses are displayed in the order in which they are received, not in the order the corresponding queries were sent." The binaries from bind use the file /etc/resolv.conf to get the nameserver to use. The file resolv.conf in the Magisk Module is: `ASUS_I006D:/ $ cat /system/etc/resolv.conf` `nameserver 8.8.8.8nameserver 1.1.1.1ASUS_I006D:/ $` To use dig on phones without the file resolv.conf use the dig parameter for the nameserver to use - for example to ask the nameserver 192.168.1.1: `ASUS_I006D:/ $ dig www.heise.de @192.168.1.1 +short193.99.144.85ASUS_I006D:/ $` To change the default nameserver, edit the file /data/adb/modules/network_tools/system/etc/resolv.conf. (Download link for the source: https://gitlab.isc.org/isc-projects/bind9/-/branches?state=all&sort=updated_desc&search=9.11)	1.0.0
configure_wlan_monitor_mode.sh	configure_wlan_monitor_mode.sh is a shell script to enable or disable monitor mode for the WLAN interface	n/a	the usage for the script is: : `ASUS_I006D:/data/adb/modules/network_tools/system/bin # cdASUS_I006D:/ # configure_wlan_monitor_mode.sh -h` `Usage: configure_wlan_monitor_mode.sh [on\|off\|enable\|disable\|status] [if=interface] [executable_to_execute] [parameter_for_the executable]` `The default interface is wlan0` `ASUS_I006D:/ #` Be aware that enabling the monitor mode disables WiFi. To check the current WiFi status use `settings get global wifi_on` To enable WiFi again after disabling the monitor mode use the command `svc wifi enable`	1.0.0

NcFTP	NcFTP is a ftp client	https://www.ncftp.com/	The configuration directory for NcFTP is /data/local/tmp/ncftp_config; to change the directory set and export the variable NCFTPDIR before calling NcFTP or edit the value for the variable NCFTPDIR in the file /data/adb/modules/network_tools/system/bin/ncftp.	1.1.0
logger, telnet, ftp	logger, telnet and tftp from the GNU inteutils 2.5 (https://www.gnu.org/software/inetutils/inetutils.html)	https://www.gnu.org/software/inetutils/inetutils.html	The name of the logger from the GNU inetutils provided by this Magisk Module is glogger. The logger from the GUN inetutils can not be used to write a message to the logs on the Android OS. But it can be used to write a log message to an external loghost. Example To write a message to the loghost 192.168.1.126 use `glogger -p user.error -h 192.168.1.126 "Hallo Ihr da draussen"`	1.1.0

ifstat	"ifstat is a tool to report network interfaces bandwith just like vmstat/iostat do for other system counters. "	http://gael.roualland.free.fr/ifstat/		1.2.0
netperf, netserver	netperf is a network benchmark tool	http://www.netperf.org/netperf		1.2.0
links	links is a text mode web browser	http://links.twibright.com		1.2.0
lynx	lynx is a text mode web browser	https://lynx.invisible-island.net/		1.2.0

Notes

The Magisk Module installs the binaries in /system/bin

Most executables are statically linked executables.

*.static files are statically linked versions of the executabes that are dynamically linked.

Note that the statically linked executables *.static cannot resolve hostnames via DNS - you can only use them with IP addresses.

Necessary libraries for the tools are installed in the directory /system/lib64/newlibs

ls -l /system/lib64/newlibs

┌shell@localhost:/ └─$ ls -l /system/lib64/newlibs/ total 10308-rwxr-xr-x 1 root root 257232 2024-10-03 14:23 libbz2.so.1.0-rwxr-xr-x 1 root root 6846664 2024-10-03 14:23 libcrypto.so.3lrwxrwxrwx 1 root root 17 1970-03-22 04:22 libiconv.so.2 -> libiconv.so.2.6.1-rwxr-xr-x 1 root root 1053856 2024-10-03 14:23 libiconv.so.2.6.1lrwxrwxrwx 1 root root 17 1970-03-22 04:22 libncurses.so.6 -> libncurses.so.6.3-rwxr-xr-x 1 root root 507376 2024-10-03 14:23 libncurses.so.6.3-rwxr-xr-x 1 root root 329560 2024-10-03 14:23 libpcap.so.1-rwxr-xr-x 1 root root 1546856 2024-10-03 14:23 libssl.so.3 ┌shell@localhost:/ └─$

For all tools, that need one of these libraries, the wrapper script /system/bin/network_tools_bin_wrapper is used:

ls -l /system/bin | grep network_tools_bin_wrapper

┌shell@localhost:/
└─$ ls -l /system/bin | grep network_tools_bin_wrapper
lrwxrwxrwx 1 root root       27 1970-03-22 06:31 bmon -> ./network_tools_bin_wrapper
lrwxrwxrwx 1 root root       27 1970-03-22 06:31 curl -> ./network_tools_bin_wrapper
lrwxrwxrwx 1 root root       27 1970-03-22 06:31 links -> ./network_tools_bin_wrapper
lrwxrwxrwx 1 root root       27 1970-03-22 06:31 lynx -> ./network_tools_bin_wrapper
lrwxrwxrwx 1 root root       27 1970-03-22 06:31 ncftp -> ./network_tools_bin_wrapper
lrwxrwxrwx 1 root root       27 1970-03-22 06:31 ncftpbatch -> ./network_tools_bin_wrapper
lrwxrwxrwx 1 root root       27 1970-03-22 06:31 ncftpget -> ./network_tools_bin_wrapper
lrwxrwxrwx 1 root root       27 1970-03-22 06:31 ncftpls -> ./network_tools_bin_wrapper
lrwxrwxrwx 1 root root       27 1970-03-22 06:31 ncftpput -> ./network_tools_bin_wrapper
lrwxrwxrwx 1 root root       27 1970-03-22 06:31 ncftpspooler -> ./network_tools_bin_wrapper
-rwxr-xr-x 1 root root     1272 2024-10-03 16:32 network_tools_bin_wrapper
lrwxrwxrwx 1 root root       27 1970-03-22 06:31 wget -> ./network_tools_bin_wrapper
lrwxrwxrwx 1 root root       27 1970-03-22 06:31 wget2 -> ./network_tools_bin_wrapper
┌shell@localhost:/
└─$

The instructions to compile wget, nmap, curl and others are taken from this script:

https://github.com/Zackptg5/Cross-Compiled-Binaries-Android/blob/master/build_script/build.sh

A big thanks to Zackptg5 -- the build instructions for various Unix tools in the build.sh script in his repo helped a lot to compile the Unix tools in Android

History of this entry

05.10.2024

initial release

Documentation for the Magisk Module with OpenSSSH

URL: https://xdaforums.com/t/module-openssh-for-android.4696499/

Documentation for the Magisk Module with OpenSSSH

The Magisk Module openssh_9.9p1.v1.0.0.zip contains the server and client binaries from OpenSSH 9.9p1 for Android running on a arm64 CPU.

The files and directories in the Module are:

FIle/Directory	content	Comment
/system/bin/openssh_bin_wrapper	wrapper script for the ssh binaries	The wrapper script is necessary to use the OpenSSH libraries
/system/usr/bin/scp /system/usr/bin/sftp /system/usr/bin/ssh /system/usr/bin/ssh-add /system/usr/bin/ssh-agent /system/usr/bin/ssh-keygen /system/usr/bin/ssh-keyscan /system/usr/bin/sshd	ssh binaries
/system/bin/ssh /system/bin/ssh-add /system/bin/ssh-agent /system/bin/ssh-copy-id /system/bin/ssh-keygen /system/bin/ssh-keyscan /system/bin/sshd	symbolic links for the ssh binaries	These symbolic links to the wrapper script /system/bin/openssh_bin_wrapper are created by the customization script
/system/lib64/openssh_libs/libcrypto.so /system/lib64/openssh_libs/libedit.so /system/lib64/openssh_libs/libldns.so /system/lib64/openssh_libs/libncurses.so /system/lib64/openssh_libs/libssl.so	libraries necessary for OpenSSH	The libraries for OpenSSH must not overwrite the libraries from the OS
/system/usr/libexec/bin/sftp-server /system/usr/libexec/bin/ssh-keysign /system/usr/libexec/bin/ssh-pkcs11-helper /system/usr/libexec/bin/ssh-sk-helper /system/usr/libexec/bin/sshd-session	helper programs for ssh
/system/usr/libexec/sftp-server /system/usr/libexec/ssh-keysign /system/usr/libexec/ssh-pkcs11-helper /system/usr/libexec/ssh-sk-helper /system/usr/libexec/sshd-session	symbolic links for the helper programs	These symbolic links to the wrapper script /system/bin/openssh_bin_wrapper are created by the customization script
/data/local/tmp/var/ /data/local/tmp/var/empty /data/local/tmp/var/log /data/local/tmp/var/mail /data/local/tmp/var/run /data/local/tmp/home	writable data directories for ssh and sshd	These directories will be created by the customization script if they do not yet exist. The data in these directories is not changed when the Magisk Module is reinstalled or updated

/data/local/tmp/home/.ssh	.ssh directory of the user starting the sshd	This directory is used for the files known_hosts, authorized_keys and others This directory is created by the customization script if it does not yet exist.

/etc/ssh/ssh_host_ed25519_key /etc/ssh/ssh_host_ed25519_key.pub /etc/ssh/ssh_host_rsa_key /etc/ssh/ssh_host_rsa_key.pub	ssh host keys	The host keys are created by the customization script of the Magisk Module if they do not yet exist The host keys are preserved if the Magisk Module is reinstalled or updated To change or add new ssh host keys, create the keys in the directory /data/adb/modules/openssh/system/etc/ssh, update the file /system/etc/sshd_config, and reboot the phone Note that the owner of the host keys must be the user shell.

/etc/ssh/ssh_config	ssh config file	To update the file, edit the file /data/adb/modules/openssh/system/etc/ssh/ssh_config and reboot the phone or remount the file /system/etc/ssh_config
/etc/ssh/sshd_config	sshd config file	To update the file, edit the file /data/adb/modules/openssh/system/etc/ssh/sshd_config and reboot the phone or remount the file /system/etc/sshd_config

/data/adb/modules_update/openssh/service.sh	start script for the sshd

The customization script creates symbolic links to the wrapper script for each executable in the module. With a wrapper script, we ensure that the executables find the additional libraries for OpenSSH.

The module service script /data/adb/modules_update/openssh/service.sh starts the sshd automatically after booting the phone if the file /data/local/tmp/home/start_sshd exists.
The sshd is started by the script with the log file /data/local/tmp/var/log/sshd.log if that file exists.
Additional parameter for the sshd started via service.sh can be defined in the file /data/local/tmp/home/.ssh/sshd_parameter.
The sshd is started as user shell.

The file /data/local/tmp/home/start_sshd is NOT created by the customization script of the Magisk Module.

Access via ssh is only possible with a public ssh key. To do this, the public ssh keys of the users who are allowed to connect must be entered in the file /data/local/tmp/home/.ssh/authorized_keys.

Notes

Please note that the sshd configuration is somehow not secure to allow the sshd to be started by the user shell!

The sshd can be started by the user shell or the user root. It's recommended to always start the sshd as user shell to not change the permissions for the data files used by sshd.
The default port used by the sshd is 9022; the port is configured in the file /etc/ssh/sshd_config:

┌shell@localhost:/ └─$ grep ^Port /etc/ssh/sshd_config Port 9022 ┌shell@localhost:/ └─$

Details for compiling OpenSSH for Android

The binaries are compiled with the Android NDK r27b using the API version 33.

OpenSSH was compiled with these options:

Compile options for OpenSSH

OpenSSH has been configured with the following options: User binaries: /system/usr/bin System binaries: /system/usr/sbin Configuration files: /system/etc/ssh Askpass program: /system/usr/libexec/ssh-askpass Manual pages: /system/usr/man/manX PID file: /data/local/tmp/var/run Privilege separation chroot path: /data/local/tmp/var/empty sshd default user PATH: /system/bin Manpage format: doc PAM support: no OSF SIA support: no KerberosV support: no SELinux support: no libedit support: yes libldns support: yes Solaris process contract support: no Solaris project support: no Solaris privilege support: no IP address in $DISPLAY hack: no Translate v4 in v6 hack: no BSD Auth support: no Random number source: OpenSSL internal ONLY Privsep sandbox style: none PKCS#11 support: yes U2F/FIDO support: yes Host: aarch64-unknown-linux-android Compiler: /data/develop/android/android-ndk-r27b/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android33-clang Compiler flags: --sysroot /data/develop/android/android-ndk-r27b/toolchains/llvm/prebuilt/linux-x86_64/sysroot --sysroot /data/develop/android/android-ndk-r27b/toolchains/llvm/prebuilt/linux-x86_64/sysroot --sysroot /data/develop/android/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/sysroot -O2 -fPIE -fPIC -I/data/develop/android/sysroot/usr/include -Wno-implicit-function-declaration -Wno-int-conversion -O2 -fPIE -fPIC -I/data/develop/android/sysroot/usr/include -Wno-implicit-function-declaration -Wno-int-conversion -O2 -fPIE -fPIC -I/data/develop/android/sysroot/usr/include -Wno-implicit-function-declaration -Wno-int-conversion -pipe -Wunknown-warning-option -Wno-error=format-truncation -Qunused-arguments -Wall -Wextra -Wpointer-arith -Wuninitialized -Wsign-compare -Wformat-security -Wsizeof-pointer-memaccess -Wno-pointer-sign -Wno-unused-parameter -Wno-unused-result -Wmisleading-indentation -Wbitwise-instead-of-logical -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -ftrapv -fzero-call-used-regs=used -ftrivial-auto-var-init=zero -mretpoline -fno-builtin-memset -Dfd_mask=int Preprocessor flags: -DHAVE_ATTRIBUTE__SENTINEL__=1 -DBROKEN_SETRESGID -I/system/usr/include Linker flags: --sysroot /data/develop/android/android-ndk-r27b/toolchains/llvm/prebuilt/linux-x86_64/sysroot --sysroot /data/develop/android/android-ndk-r27b/toolchains/llvm/prebuilt/linux-x86_64/sysroot --sysroot /data/develop/android/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/sysroot -s -pie -L/data/develop/android/sysroot/usr/lib -s -pie -L/data/develop/android/sysroot/usr/lib -s -pie -L/data/develop/android/sysroot/usr/lib -Wl,-z,relro -Wl,-z,now -Wl,-z,noexecstack -Wl,-z,retpolineplt -L/system/usr/lib Libraries: -lldns +for channels: -lcrypto -lz [xtrnaw7@t15g /data/develop/android/source/openssh-9.9p1]$

The configure options used are copied from the Termux Build script for OpenSSH:

    https://github.com/termux/termux-packages/blob/master/packages/openssh/build.sh

The configure command used to create the config for compiling OpenSSH in Android is in the file

   source/myconfigure

in the Magisk Module

All patches for OpenSSH from Termux that are useful for a Magisk Module with OpenSSH are applied to the source code.

The patches for OpenSSH in Termux are available here:

    https://github.com/termux/termux-packages/tree/master/packages/openssh

The patches used are in the directory ./source/patches in the Magisk Module; the patches were copied from the Termux repository at 2024-10-04.
The script source/patches/apply_patches.sh in the ZIP file with the Magisk Module can be used to apply the patches.

An additional patch was necessary to get the OpenSSH binaries working for the user shell:

Source: https://stackoverflow.com/questions/76376942/openssh-9-3p-fails-to-build-for-android
(This is the patch misc.c.additional.patch in the directory source/patches in this Magisk Module)

Trouble Shooting

To execute the customize script of the Magisk Module with "set -x" and write STDOUT and STDERR to the file /data/local/tmp/openssh_customize.log create the (probably empty) file /data/local/tmp/debug before installing the Magisk Module.

To execute the service script to start the sshd with "set -x" and write STDERR and STDLOG messages to the file /data/local/tmp/start_sshd.log create the file /data/local/tmp/debug before rebooting the phone.

History of this entry

26.04.2024

initial release

Documentation for the Magisk Module with Python 3.14

URL: https://xdaforums.com/t/module-python-3-14-for-android.4696862/

Documentation for the Magisk Module with Python 3.14

The Magisk Module python3_3.14.v1.0.0.zip contains Python3 3.14 compiled for Android running on a arm64 CPU.

shell@localhost:/└─$ python --version -vPython 3.14.0a0┌shell@localhost:/└─$

The configured directories for site-packages in this Python installation are:

shell@localhost:/└─$ python3 -m sitesys.path = [    '/',    '/system/usr/lib/python314.zip',    '/system/usr/lib/python3.14',    '/system/usr/lib/python3.14/lib-dynload',    '/system/usr/lib/python3.14/site-packages',    '/data/local/tmp/home/python3.14/site-packages',    '/data/local/tmp/home/python/site-packages',]USER_BASE: '/data/local/tmp/home/.local' (doesn't exist)USER_SITE: '/data/local/tmp/home/.local/lib/python3.14/site-packages' (doesn't exist)ENABLE_USER_SITE: True┌shell@localhost:/└─$

The directories

/data/local/tmp/home/python3.14/site-packages
/data/local/tmp/home/python/site-packages

are writable by the user shell and can be used to add additional site-packages.

All Python executables are executed via a wrapper script:

┌shell@localhost:/└─$ ls -l /system/bin/ | grep python_wrapperlrwxrwxrwx 1 root root       16 1970-03-25 22:41 idle3 -> ./python_wrapperlrwxrwxrwx 1 root root       16 1970-03-25 22:41 pydoc3 -> ./python_wrapperlrwxrwxrwx 1 root root       16 1970-03-25 22:41 python3 -> ./python_wrapperlrwxrwxrwx 1 root root       16 1970-03-25 22:41 python3-config -> ./python_wrapper-rwxr-xr-x 1 root root     1241 2024-10-07 08:41 python_wrapper┌shell@localhost:/└─$

The wrapper script for the Python executables executes the script

/data/local/tmp/home/python_init.sh

before executing the "real" python executable if it exists.

This file can be used to define environment variables for Python - see https://docs.python.org/3/using/cmdline.html for a list of supported environment variables ( ... or do anything else)
(use the variable "$0" in the script to distinguish between the executables)

All libraries for Python are in /system/usr/lib :

┌root@localhost:/└─# ls -ld /system/usr/lib/*python*lrwxrwxrwx 1 root root       20 1970-03-26 07:48 /system/usr/lib/libpython3.14.so -> libpython3.14.so.1.0-rw-r--r-- 1 root root 43871608 2024-10-07 17:46 /system/usr/lib/libpython3.14.so.1.0-rw-r--r-- 1 root root     6712 2024-10-07 17:46 /system/usr/lib/libpython3.sodrwxr-xr-x 40 root root     3880 1970-03-26 07:48 /system/usr/lib/python3.14┌root@localhost:/└─#

Details for compiling Python3 for Android

The binaries are compiled with the gcc toolchain from the Android NDK r17c using the API version 21 on a phone runing Android 14.

Python3 has been compiled with these configure options (copied from the Termux repository entry for Python3)

export CFLAGS="$CFLAGS -D__ANDROID_API__=21 -O3"export CPPFLAGS="$CFLAGS"export TARGET_ROOT=/data/local/tmp/python3.14AC_OPTIONS="ac_cv_func_shm_unlink=no ac_cv_func_shm_open=yesac_cv_posix_semaphores_enabled=yes ac_cv_func_linkat=no ac_cv_func_faccessat=no ac_cv_file__dev_ptc=no"eval ${AC_OPTIONS} ./configure --prefix=${TARGET_ROOT}/usr --with-openssl=/data/local/tmp/develop/sysroot/usr --enable-loadable-sqlite-extensions --enable-shared --enable-pystats --with-builtin-hashlib-hashes=md5,sha1,sha2,sha3,blake2 \ --with-system-expat --without-ensurepip --with-tzpath=/system/usr/share/zoneinfo

Result:

The necessary bits to build these optional modules were not found: _ctypes _ctypes_test _dbm _lzma _posixshmem _tkinter grp readline To find the necessary bits, look in configure.ac and config.log. Checked 112 modules (34 built-in, 69 shared, 1 n/a on linux-aarch64, 0 disabled, 8 missing, 0 failed on import)

All approbiate patches from the Termux repository for Python

https://github.com/termux/termux-packages/tree/master/packages/python

are applied.

The patches applied are in the directory source/patches in the Magisk Module.

Notes

The cache directory for compiled Python scripts is defined in the wrapper script for the Python executables

PYTHONPYCACHEPREFIX="/data/local/tmp/home/python/cache"

The Python executables in this Magisk Module are compiled on an ASUS Zenfone 8 with the GCC Toolchain (version 1.0.0) from this Magisk Module:

https://github.com/Googlers-Repo/gcc/

shell@localhost:/└─$ file /system/usr/bin/python3.14 /system/usr/bin/python3.14: ELF shared object, 64-bit LSB arm64, dynamic (/system/bin/linker64), for Android 21, built by NDK r17c (4988734), not stripped┌shell@localhost:/└─$

----

History of this entry

07.10.2024

initial release

Links

Files mentioned in the HowTos

Show or Hide Table

File	Date / Version	Mentioned in	Comment
install_apk.sh	04.08.2024 2.0.0	How to install packages (apk files) for Android via script	the script can run on the PC or in an adb session on the phone
install_magisk_via_twrp.sh	01.05.2024 v3.1.3.3	How to install Magisk into the boot partition using script
PlayStore_for_MicroG.zip		How to replace the Fake Store from OmniROM with MicroG with a patched Playstore	the Playstore version is 20-9-20-all
initshell.zip		How to change the home directory for the user root on an Android phone
profile		How to change the home directory for the user root on an Android phone
bootctl-binary-v2.1.3.zip	2.1.3	How to switch the active slot	The original repository https://github.com/roihershberg/bootctl-binary is not maintained anymore by the author.
install_twrp_from_within_twrp.sh	30.09.2023 1.2.0.0	How to install TWRP via script
install_twrp.sh	28.01.2023 1.3.1.0	How to install TWRP via script
boot_phone_from_twrp.sh	13.09.2024 v3.1.6.0	How to boot a phone from a TWRP recovery image via script How to enable root access using Magisk in a script
enable_wifi.sh	20.11.2022	How to change files in Android XML Binary format
sqlite3	3.39.3 2022-09-05 11:02:23	How to enable root access using Magisk in a script	static linked sqlite3 binary for 64 Bit ARM CPUs
enable_root_access_via_magisk.sh	05.02.2024 1.5.0.0	How to enable root access using Magisk in a script
init_magisk_db.sh	30.04.2024 v1.0.3.0	How to enable root access using Magisk in a script
enable_adb.sh	16.12.2022 1.0.0.0
install_os_via_twrp.sh	10.08.2024 v2.1.3.0	How to install an OS image using the TWRP binary twrp
enable_access_via_adb.sh	30.04.2024 v1.3.0.0	How to enable root access using Magisk in a script
enable_adb_using_magisk.sh	23.05.2024 1.2.3.0	How to enable root access using Magisk in a script
execute_tasks.sh	08.05.2024 v2.3.2	How to install and configure the Android OS
prepare_phone.include	22.05.2024	How to install and configure the Android OS
prepare_phone.conf	22.05.2024	How to install and configure the Android OS
install_twrp_ramdisk.sh	15.05.2023 1.0.0.0	Installing TWRP into the boot partition again after the installation of an OS update
disable_fake_store.zip	04.06.2023 1.0.0.0	How to install a more recent version of the Playstore in OmniROM
get_aosp_patches.sh	07.02.2024 1.1.0.0	How to apply the Android Security patches to a local repository
rename_apk.sh	05.02.2024 1.0.0.0	How to get the application name and version from an apk file
/files/public/Android/binaries_for_arm64/		Examples for compiling C programs for Android using gcc or clang	Binaries compiled for arm64 Filelist: [aarch64-linux-android] [xtrnaw7@t15g /data/www/myhomepage/htdocs/files/public/Android/binaries_for_arm64]$ date Mon Sep 30 07:49:35 AM CEST 2024 [aarch64-linux-android] [xtrnaw7@t15g /data/www/myhomepage/htdocs/files/public/Android/binaries_for_arm64]$ ls -l total 16760 -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 1777096 Sep 7 13:36 awk -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 268576 Feb 19 2024 bc -rwxrwxr-x. 1 xtrnaw7 xtrnaw7 628616 Aug 12 12:55 bmore_1.4.2_static -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 408680 Feb 19 2024 bvi -rwxrwxr-x. 1 xtrnaw7 xtrnaw7 824328 Aug 12 12:55 bvi_1.4.2_static -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 233352 Feb 19 2024 cpio -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 7260120 Aug 25 08:26 dig -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 486456 Feb 19 2024 less -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 13000 Feb 19 2024 lessecho -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 87040 Aug 25 08:28 ngrep -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 732936 Feb 19 2024 nmon -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 144256 Feb 19 2024 pigz -rwxrwxr-x. 1 xtrnaw7 xtrnaw7 3666232 Aug 12 12:55 vim_9.1_static -rwxrwxr-x. 1 xtrnaw7 xtrnaw7 392248 Aug 12 12:56 xxd_9.1_static -rw-r--r--. 1 xtrnaw7 xtrnaw7 211016 Sep 29 08:54 xz [aarch64-linux-android] [xtrnaw7@t15g /data/www/myhomepage/htdocs/files/public/Android/binaries_for_arm64]$
print_security_patch	09.04.2024 1.1.0.0	Infos for building a CustomROM image How to print the security patch level used in repositories or ROM image zip files
check_apks.sh	10.04.2024 1.0.0.0	How to fix a hanging restore in Titanium Backup
ASUS_Zenfone8_prog_firehose_ddr.elf	Binary build date: Aug 13 2021 @ 23:00:54	Some hints about EDL mode	firehose loader for edl mode on the ASUS Zenfone 8
02A firehose.zip	Binary build date: Mar 23 2022 @ 23:27:20	Some hints about EDL mode	zip file with some firehose loader for EDL mode on the ASUS Zenfone 8
usbreset		How to reset the USB port used to connect an Android phone	small Linux program to reset an USB port Source code: `https://askubuntu.com/questions/645/how-do-you-reset-a-usb-device-from-the-command-line` or download the copy of the script from here: usbreset.c If the binary does not work on your Linux OS, please download the source code for the program and recompile it using these commands: `cc usbreset.c -o usbreset chmod 755 usbreset`
reset_usb_port_for_phone.sh	1.0.1	How to reset the USB port used to connect an Android phone
enable_adb_in_recovery_ramdisk.sh	26.06.2024 v1.0.1	How to add files to the ramdisk used for the recovery boot
create_ugly_twrp_image.sh	27.06.2024 v1.0.0	How to create a TWRP image for LineageOS based ROMs using a script
list_bind_mounts.sh	28.07.2024 v1.0.0	How to list bind mounts
apply_security_level_to_StatixOS_repos.sh	12.08.2024 v1.0.0	How to apply an Android patchlevel to the repositories for the StatiXOS
sign_apk.sh	28.08.2024 1.0.0	How to re-sign an apk file
install_mm.sh	21.09.2024 1.1.0	Some hints for creating Magisk Modules How to install Magisk modules via a script
remount_dynamic_partitions.sh	07.09.2024 1.1.0	How to create logical devices using the device mapper from Android How to access the system partitions while the phone is booted into the LineageOS recovery
list_logical_device_backends.sh	07.09.2024 1.1.0	How to create logical devices using the device mapper from Android
list_logical_device_usage.sh	03.09.2024 1.0.0	How to create logical devices using the device mapper from Android

Other Magisk Modules

Show or Hide Table

For questions regarding these modules please reply to this post in the XDA forums:

https://xdaforums.com/t/module-magisk-modules-with-unix-binaries-compiled-for-the-android-os.4687095/

File	Content	Comment
btop-1.3.2.1.zip	Magisk Module with btop binaries for Android for these CPUs: aarch64 / arm64 / armv5l / armv7l / i486 / i686 / mips64 / powerpc64 / x86_64	the binaries in the Magisk Module are static linked binaries from https://github.com/aristocratos/btop/releases Note that btop for Android needs root permissions. The home page for btop is https://github.com/aristocratos/btop
ckermit_10.0_Beta10.zip	Magisk Module with ckermit binaries for Android for these CPUs: aarch64 / arm64 / armv5l / armv7l / i386 / mips / mips64 /x86_64	The source code and build instructions for ckermit are available here: https://kermitproject.org/ckdaily.html The home page for ckermit is here: https://kermitproject.org/
androidsdk34_0_3_34.0.3.01.zip	Magisk Module with the Android SDK binaries for devices with an aarch64 (= arm64)	The binaries are from this zip file: https://github.com/lzhiyong/android-sdk-tools/releases/download/34.0.3/android-sdk-tools-static-aarch64.zip
selinux_tools_v1.3.zip	Magisk module with the SELinux tools for various CPUs	the binaries in the Magisk Module are copied from this repository: https://github.com/xmikos/setools-android The files in this module are: seinfo sesearch sepolicy-inject compiled for the Android OS. The module contains binaries for these CPUs: armeabi / armeabi-v7a / arm64-v8a / mips64 / x86 / x86_64
selinux_tools_v1.4.zip	Magisk module with the SELinux tools for various CPUs	the binaries in the Magisk Module for all CPUs are copied from this repository: https://github.com/xmikos/setools-android The additional files for arm/arm64 CPUs are from this repository: https://github.com/SELinuxProject/selinux The files in this module are: seinfo sesearch sepolicy-inject compiled for the Android OS for these CPUs: armeabi / armeabi-v7a / arm64-v8a / mips64 / x86 / x86_64 There are also these binaries For arm64 and arm CPUs in the module: chkcon sepol_check_access sepol_compute_av sepol_compute_member sepol_compute_relabel sepol_validate_transition
vim_9.1.zip	Magisk Module with vim 9.1 for arm64 CPUs	The vim in the module is dynamic linked; therefore the module also install these libaries: ls -l /data/adb/modules/vim/system/lib64 # ls -l /data/adb/modules/vim/system/lib64/ total 1720 lrwxrwxrwx 1 root root 20 2024-07-26 14:24 libcanberra.so.0 -> libcanberra.so.0.2.5 -rwxr-xr-x 1 root root 54792 2024-07-26 14:24 libcanberra.so.0.2.5 -rwxr-xr-x 1 root root 1067696 2024-07-26 14:24 libiconv.so -rwxr-xr-x 1 root root 36560 2024-07-26 14:24 libltdl.so lrwxrwxrwx 1 root root 18 2024-07-26 14:24 libncursesw.so.6 -> libncursesw.so.6.4 -rwxr-xr-x 1 root root 387592 2024-07-26 14:24 libncursesw.so.6.4 -rwxr-xr-x 1 root root 26416 2024-07-26 14:24 libogg.so -rwxr-xr-x 1 root root 151328 2024-07-26 14:24 libvorbis.so -rwxr-xr-x 1 root root 28600 2024-07-26 14:24 libvorbisfile.so # The vim uses this directory for data and configs /data/local/tmp/share/vim/vim91 The default config file used by the vim is /data/local/tmp/share/vim/vim91/defaults.vim
vim_9.1.672.zip	Magisk Module with vim 9.1.672 for arm64 CPUs	The files in this Magisk Module are installed in the directory /system_ext/bin; the Magisk Module contains static and dynamic linked binaries: `ASUS_I006D:/ $ ls -l /system_ext/bin/vim.* lrwxrwxrwx 1 root root 13 1970-01-29 00:15 /system_ext/bin/vim.bin -> ./vim.dynamic -rwxr-xr-x 1 root root 3553592 2024-08-12 10:17 /system_ext/bin/vim.dynamic -rwxr-xr-x 1 root root 3666232 2024-08-12 10:17 /system_ext/bin/vim.static ASUS_I006D:/ $` The data files for vim are installed in the directory /data/local/tmp/share/vim/vim91. vim --version ASUS_I006D:/ $ vim --version VIM - Vi IMproved 9.1 (2024 Jan 02, compiled Aug 12 2024 09:27:34) Included patches: 1-672 Compiled by bnsmb Huge version without GUI. Features included (+) or not (-): +acl +file_in_path +mouse_urxvt -tag_any_white +arabic +find_in_path +mouse_xterm -tcl +autocmd +float +multi_byte +termguicolors +autochdir +folding +multi_lang +terminal -autoservername -footer -mzscheme +terminfo -balloon_eval +fork() +netbeans_intg +termresponse +balloon_eval_term -gettext +num64 +textobjects -browse -hangul_input +packages +textprop ++builtin_terms -iconv +path_extra +timers +byte_offset +insert_expand -perl +title +channel +ipv6 +persistent_undo -toolbar +cindent +job +popupwin +user_commands -clientserver +jumplist +postscript +vartabs -clipboard +keymap +printer +vertsplit +cmdline_compl +lambda +profile +vim9script +cmdline_hist +langmap -python +viminfo +cmdline_info +libcall -python3 +virtualedit +comments +linebreak +quickfix +visual +conceal +lispindent +reltime +visualextra +cryptv +listcmds +rightleft +vreplace +cscope +localmap -ruby +wildignore +cursorbind -lua +scrollbind +wildmenu +cursorshape +menu +signs +windows +dialog_con +mksession +smartindent +writebackup +diff +modify_fname -sodium -X11 +digraphs +mouse -sound +xattr -dnd -mouseshape +spell -xfontset -ebcdic +mouse_dec +startuptime -xim +emacs_tags -mouse_gpm +statusline -xpm +eval -mouse_jsbterm -sun_workshop -xsmp +ex_extra +mouse_netterm +syntax -xterm_clipboard +extra_search +mouse_sgr +tag_binary -xterm_save -farsi -mouse_sysmouse -tag_old_static system vimrc file: "$VIM/vimrc" user vimrc file: "$HOME/.vimrc" 2nd user vimrc file: "~/.vim/vimrc" 3rd user vimrc file: "~/.config/vim/vimrc" user exrc file: "$HOME/.exrc" defaults file: "$VIMRUNTIME/defaults.vim" fall-back for $VIM: "/data/local/tmp/share/vim" Compilation: /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 -std=gnu11 -c -I. -Iproto -DHAVE_CONFIG_H --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-21/arch-arm64 -O2 -fPIE -fPIC -I/data/develop/android/sysroot/usr/include -I/data/develop/android/android-ndk-r14b/platforms/android-21/arch-arm64/usr/include --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-21/arch-arm64 -O2 -fPIE -fPIC -I/data/develop/android/sysroot/usr/include -I/data/develop/android/android-ndk-r14b/platforms/android-21/arch-arm64/usr/include -D_REENTRANT -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 Linking: /data/develop/android/android-ndk-r14b/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-gcc-4.9 -std=gnu11 --sysroot=/data/develop/android/android-ndk-r14b/platforms/android-21/arch-arm64 -s -pie -L/data/develop/android/sysroot/usr/lib -L/data/develop/android/android-ndk-r14b/platforms/android-21/arch-arm64/usr/lib -Wl,--as-needed -o vim -lm -lncurses ASUS_I006D:/ $ To extract only the static linked binary from the ZIP file use this command: `unzip -p vim_9.1.672.zip system/system_ext/bin/vim.static >./vim.static` see Examples for compiling C programs for Android using gcc or clang for instructions to compile vim
DebugTools_9.1.2.0.zip	Magisk Module with some Linux trouble shooting tools for arm64 CPUs	List of files in the module [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/ModuleSrc/DebugTools/system/bin ] $ date Tue Aug 6 02:56:10 PM CEST 2024 [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/ModuleSrc/DebugTools/system/bin ] $ ls -ltr total 10788 -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 87040 Jul 26 16:18 ngrep -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 4871448 Jul 26 16:18 ncat -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 813096 Jul 26 16:18 strace -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 3661856 Jul 26 16:18 tcpdump -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 541460 Jul 26 16:19 fdisk -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 680612 Jul 26 16:19 gdisk -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 51 Jul 26 16:43 sudo -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 4083 Aug 1 17:06 list_bind_mounts.sh -rwxrwxr-x. 1 xtrnaw7 xtrnaw7 13096 Aug 6 14:29 bgrep -rwxrwxr-x. 1 xtrnaw7 xtrnaw7 73776 Aug 6 14:41 hexdump -r-xr-xr-x. 1 xtrnaw7 xtrnaw7 190576 Aug 6 14:44 bvi.bin -r-xr-xr-x. 1 xtrnaw7 xtrnaw7 67328 Aug 6 14:44 bmore.bin -r-xr-xr-x. 1 xtrnaw7 xtrnaw7 55 Aug 6 14:47 bvi -r-xr-xr-x. 1 xtrnaw7 xtrnaw7 55 Aug 6 14:47 bmore [ OmniRom 14 Dev - xtrnaw7@t15g /data/develop/android/ModuleSrc/DebugTools/system/bin ] $
bvi_1.4.1.zip	Magisk Module with bvi for arm64 CPUs	bvi is an editor for binary files with the user interface from vi The source code for bvi is available here: https://bvi.sourceforge.net/ The executable in this module is dynamic linked; the necessasry library libncursesw.so.6 is also in the module.
bvi_1.4.2.zip	Magisk Module with bvi for arm64 CPUs	bvi is an editor for binary files with the user interface from vi The source code for bvi is available here: https://bvi.sourceforge.net/ The executable in this module is static linked see Examples for compiling C programs for Android using gcc or clang for instructions to compile bvi
coreutils_9.5.0.0.zip	Magisk Module with the Linux core utils for arm64 CPUs	The source for the files in this Magisk Module is available here: https://github.com/coreutils/coreutils/ The configure statement used to compile these tools was: `./configure --prefix=/data/local/tmp CFLAGS="-Wno-implicit-function-declaration -Wno-int-conversion" --with-openssl=no --without-libgmp` -> the tools only use the standard libraries from Android The tools are installed in the directory /system/bin/coreutils List of executables in the module as of 28.07.2024: [ b2sum base32 base64 basename basenc cat chcon chgrp chmod chown chroot cksum comm cp csplit cut date dcgen dd df dir dircolors dirname du du-tests echo env expand expr factor false fmt fold getlimits ginstall groups head id join kill libstdbuf.so link ln logname ls make-prime-list md5sum mkdir mkfifo mknod mktemp mv nice nl nohup nproc numfmt od paste pathchk pinky pr printenv printf ptx pwd readlink realpath rm rmdir runcon seq sha1sum sha224sum sha256sum sha384sum sha512sum shred shuf sleep sort split stat stdbuf stty sum sync tac tail tee test timeout touch tr true truncate tsort tty uname unexpand uniq unlink uptime users vdir wc who whoami yes
myscripts_1.1.0.0.zip	A Magisk module with some of my scripts for Android	Files in the module: ASUS_I006D:/data/adb/modules/myscripts/system/bin # date Sun Sep 8 15:50:29 CEST 2024 ASUS_I006D:/data/adb/modules/myscripts/system/bin # ls -l total 112 -rwxr-xr-x 1 root root 156 2024-09-08 15:49 0001logcatboot -rwxr-xr-x 1 root root 354 2024-09-08 15:49 correct_dev_pn553.sh -rwxr-xr-x 1 root root 51 2024-09-08 15:49 disable_intent_filter_verification.sh -rwxr-xr-x 1 root root 146 2024-09-08 15:49 enable_microg.sh -rwxr-xr-x 1 root root 2956 2024-09-08 15:49 enable_wifi.sh -rwxr-xr-x 1 root root 11864 2024-09-08 15:49 enable_wireless_adb.sh -rwxr-xr-x 1 root root 10108 2024-09-08 15:49 install_apk.sh -rwxr-xr-x 1 root root 4083 2024-09-08 15:49 list_bind_mounts.sh -rwxr-xr-x 1 root root 7271 2024-09-08 15:49 list_logical_device_backends.sh -rwxr-xr-x 1 root root 9463 2024-09-08 15:49 list_logical_device_usage.sh -rwxr-xr-x 1 root root 223 2024-09-08 15:49 list_magisk_root_permissions.sh -rwxr-xr-x 1 root root 449 2024-09-08 15:49 list_magisk_settings.sh -rwxr-xr-x 1 root root 16866 2024-09-08 15:49 remount_dynamic_partitions.sh -rwxr-xr-x 1 root root 69 2024-09-08 15:49 remove_screenlock.sh -rwxr-xr-x 1 root root 2708 2024-09-08 15:49 rename_apk.sh -rwxr-xr-x 1 root root 42 2024-09-08 15:49 setpath.sh -rwxr-xr-x 1 root root 13 2024-09-08 15:49 sudo ASUS_I006D:/data/adb/modules/myscripts/system/bin #
openssl_3.3.1.zip	Magisk Module with OpenSSL 3.3.1 for arm64 CPUs	The OpenSSL files are installed in the directory /system/usr/openssl; a wrapper script to call openssl is installed in /system/bin. example usage: To print the certificate used for the running OS execute this command in an (adb) shell on the phone: `unzip -p /system/etc/security/otacerts.zip *pem \| openssl x509 -noout -text`
perl538_5.38.0.zip	Magisk Module with Perl 5.38 for arm64 CPUs	The Magisk Module installs Perl in the directory /data/local/tmp/perl538 The Module also install Perl wrapper scripts in the directory /system/bin: ls -l /system/bin \| grep perl ASUS_I006D:/ # ls -l /system/bin \| grep perl lrwxrwxrwx 1 root root 14 1970-01-25 03:12 cpan -> ./perl_wrapper lrwxrwxrwx 1 root root 14 1970-01-25 03:12 perl -> ./perl_wrapper -rwxr-xr-x 1 root root 192 2024-08-08 13:13 perl_env -rwxr-xr-x 1 root root 104 2024-08-08 13:13 perl_wrapper lrwxrwxrwx 1 root root 14 1970-01-25 03:12 perlbug -> ./perl_wrapper lrwxrwxrwx 1 root root 14 1970-01-25 03:12 perldoc -> ./perl_wrapper ASUS_I006D:/ # The script to start Perl defines these variables: cat /system/bin/perl_env `\|ASUS_I006D:/ # cat /system/bin/perl_env export PATH=/data/local/tmp/perl538:${PATH} export LD_LIBRARY_PATH=/data/local/tmp/perl538/lib64: export HOME="/data/local/tmp/perl_home" export PERL5LIB="/data/local/tmp/perl_home/perl5" ASUS_I006D:/ #` see How to compile Perl for Android for instructions to compile perl
perl540_5.40.0.zip	Magisk Module with Perl 5.40 for arm64 CPUs	The Magisk Module installs Perl in the directory /data/local/tmp/perl540 The Module also install Perl wrapper scripts in the directory /system/bin: ls -l /system/bin \| grep perl ASUS_I006D:/ # ls -l /system/bin \| grep perl lrwxrwxrwx 1 root root 14 1970-01-25 03:12 cpan -> ./perl_wrapper lrwxrwxrwx 1 root root 14 1970-01-25 03:12 perl -> ./perl_wrapper -rwxr-xr-x 1 root root 192 2024-08-08 13:13 perl_env -rwxr-xr-x 1 root root 104 2024-08-08 13:13 perl_wrapper lrwxrwxrwx 1 root root 14 1970-01-25 03:12 perlbug -> ./perl_wrapper lrwxrwxrwx 1 root root 14 1970-01-25 03:12 perldoc -> ./perl_wrapper ASUS_I006D:/ # The script to start Perl defines these variables: cat /system/bin/perl_env `\|ASUS_I006D:/ # cat /system/bin/perl_env export PATH=/data/local/tmp/perl538:${PATH} # LD_LIBRARY_PATH is not required for this Perl binary #export LD_LIBRARY_PATH=/data/local/tmp/perl538/lib64: export HOME="/data/local/tmp/perl_home" export PERL5LIB="/data/local/tmp/perl_home/perl5" ASUS_I006D:/ #` see Examples for compiling C programs for Android using gcc or clang for instructions to compile perl These binaries are compiled using the Android SDK; the binary only uses the default Android system libraries: ldd /data/local/tmp/perl540/bin/perl `ASUS_I006D:/ $ /data/local/tmp/perl540/bin/perl --versionThis is perl 5, version 40, subversion 0 (v5.40.0) built for linux-android` `Copyright 1987-2024, Larry Wall` `Perl may be copied only under the terms of either the Artistic License or theGNU General Public License, which may be found in the Perl 5 source kit.` `Complete documentation for Perl, including FAQ lists, should be found onthis system using "man perl" or "perldoc perl". If you have access to theInternet, point your browser at https://www.perl.org/, the Perl Home Page.` `ASUS_I006D:/ $ ldd /data/local/tmp/perl540/bin/perl` `linux-vdso.so.1 => [vdso] (0x73fc5be000)` `libdl.so => /apex/com.android.runtime/lib64/bionic/libdl.so (0x73f769b000)` `libm.so => /apex/com.android.runtime/lib64/bionic/libm.so (0x73f78c3000)` `libc.so => /apex/com.android.runtime/lib64/bionic/libc.so (0x73f770c000)` `liblog.so => /system/lib64/liblog.so (0x73f76f0000)` `libc++.so => /system/lib64/libc++.so (0x73f7808000)` `ASUS_I006D:/ $`
network_tools_1.2.0.zip	Magisk Module with various network tools for arm64 CPUs	see here for details
syslogd_2.5.0.zip	Magisk Module with the binaries syslogd and logger for arm64 CPUs from the GNU inetutils 2.5	The source code for the files in this module is available here: https://www.gnu.org/software/inetutils/inetutils.html Files in this module ASUS_I006D:/data/adb/modules/syslogd/system/bin # ls -ld $( find . ) drwxr-xr-x 3 root root 3452 2024-09-02 13:04 . -rwxr-xr-x 1 root root 51800 2024-09-02 13:04 ./nlogger lrwxrwxrwx 1 root root 15 2024-09-02 13:04 ./restart_syslogd -> ./start_syslogd drwxr-xr-x 2 root root 3452 2024-09-02 13:04 ./server -rwxr-xr-x 1 root root 79080 2024-09-02 13:04 ./server/syslogd -rwxr-xr-x 1 root root 5821 2024-09-02 13:04 ./start_syslogd lrwxrwxrwx 1 root root 15 2024-09-02 13:04 ./stop_syslogd -> ./start_syslogd ASUS_I006D:/data/adb/modules/syslogd/system/bin # see here for a description of the files in this module
ro2rw_3.7.3.0.zip	This Magisk Module contains the binaries for working with dynamic partitions in Android for arm and arm64 CPUs.	The binaries are copied from the RO2RW tools -- see here: https://xdaforums.com/t/magisk-twrp-arm32-64-a8-universal-read-only-to-read-write-for-android-ro2rw-auto-converting-super-system-partitions-to-read-write-mode.4521131/ The files can be downloaded from SourceForge: https://sourceforge.net/projects/multi-function-patch/
e2fsprogs_1.47.1.zip	This Magisk Module contains the e2fsprogs executables for arm64 CPUs	The source code for the files in this module is available here: https://e2fsprogs.sourceforge.net/ The module installs the binaries in the directory /system/bin/e2fsprogs. Files in the module ASUS_I006D:/system/bin/e2fsprogs $ ls -l total 17000 -rwxr-xr-x 1 root root 744416 2024-09-02 08:16 badblocks -rwxr-xr-x 1 root root 503088 2024-09-02 08:16 blkid -rwxr-xr-x 1 root root 466984 2024-09-02 08:16 chattr -rwxr-xr-x 1 root root 1050312 2024-09-02 08:16 debugfs -rwxr-xr-x 1 root root 837264 2024-09-02 08:16 dumpe2fs -rwxr-xr-x 1 root root 735616 2024-09-02 08:16 e2freefrag -rwxr-xr-x 1 root root 1437248 2024-09-02 08:16 e2fsck -rwxr-xr-x 1 root root 841552 2024-09-02 08:16 e2image -rwxr-xr-x 1 root root 948488 2024-09-02 08:16 e2label -rwxr-xr-x 1 root root 837264 2024-09-02 08:16 e2mmpstatus -rwxr-xr-x 1 root root 746880 2024-09-02 08:16 e2undo -rwxr-xr-x 1 root root 476336 2024-09-02 08:16 e4crypt -rwxr-xr-x 1 root root 747488 2024-09-02 08:16 e4defrag -rwxr-xr-x 1 root root 465424 2024-09-02 08:16 filefrag -rwxr-xr-x 1 root root 948488 2024-09-02 08:16 findfs -rwxr-xr-x 1 root root 509832 2024-09-02 08:16 fsck lrwxrwxrwx 1 root root 8 1970-02-18 22:14 fsck.ext2 -> ./e2fsck lrwxrwxrwx 1 root root 8 1970-02-18 22:14 fsck.ext3 -> ./e2fsck lrwxrwxrwx 1 root root 8 1970-02-18 22:14 fsck.ext4 -> ./e2fsck -rwxr-xr-x 1 root root 494080 2024-09-02 08:16 logsave -rwxr-xr-x 1 root root 469424 2024-09-02 08:16 lsattr -rwxr-xr-x 1 root root 950240 2024-09-02 08:16 mke2fs lrwxrwxrwx 1 root root 8 1970-02-18 22:14 mkfs.ext2 -> ./mke2fs lrwxrwxrwx 1 root root 8 1970-02-18 22:14 mkfs.ext3 -> ./mke2fs lrwxrwxrwx 1 root root 8 1970-02-18 22:14 mkfs.ext4 -> ./mke2fs -rwxr-xr-x 1 root root 454568 2024-09-02 08:16 mklost+found -rwxr-xr-x 1 root root 795616 2024-09-02 08:16 resize2fs -rwxr-xr-x 1 root root 948488 2024-09-02 08:16 tune2fs -rwxr-xr-x 1 root root 479568 2024-09-02 08:16 uuidd -rwxr-xr-x 1 root root 472784 2024-09-02 08:16 uuidgen ASUS_I006D:/system/bin/e2fsprogs $
python3_3.10.4.1.zip	This Magisk Module contains the Python3 binaries and files for arm and arm64 CPUs from the TWRP repository	The Magisk Module installs the binaries in /system/bin. The files in this module are available here: https://github.com/TeamWin/android_external_python3/tree/android-12.1 Files in the module ┌root@localhost:/data/adb/modules/python3 └─# find system/ system/ system/bin system/bin/python3 system/bin/python system/etc system/etc/python system/etc/python/lib system/etc/python/lib/python3.10 system/etc/python/lib/python3.10/codecs.py system/etc/python/lib/python3.10/_collections_abc.py system/etc/python/lib/python3.10/os.py system/etc/python/lib/python3.10/_weakrefset.py system/etc/python/lib/python3.10/functools.py system/etc/python/lib/python3.10/sre_constants.py system/etc/python/lib/python3.10/struct.py system/etc/python/lib/python3.10/contextlib.py system/etc/python/lib/python3.10/warnings.py system/etc/python/lib/python3.10/enum.py system/etc/python/lib/python3.10/_sysconfigdata__linux_aarch64-linux-android.py system/etc/python/lib/python3.10/collections system/etc/python/lib/python3.10/collections/abc.py system/etc/python/lib/python3.10/collections/__init__.py system/etc/python/lib/python3.10/abc.py system/etc/python/lib/python3.10/urllib system/etc/python/lib/python3.10/urllib/robotparser.py system/etc/python/lib/python3.10/urllib/parse.py system/etc/python/lib/python3.10/urllib/error.py system/etc/python/lib/python3.10/urllib/request.py system/etc/python/lib/python3.10/urllib/__init__.py system/etc/python/lib/python3.10/urllib/response.py system/etc/python/lib/python3.10/typing.py system/etc/python/lib/python3.10/types.py system/etc/python/lib/python3.10/genericpath.py system/etc/python/lib/python3.10/fnmatch.py system/etc/python/lib/python3.10/pathlib.py system/etc/python/lib/python3.10/heapq.py system/etc/python/lib/python3.10/posixpath.py system/etc/python/lib/python3.10/sre_parse.py system/etc/python/lib/python3.10/xml system/etc/python/lib/python3.10/xml/dom system/etc/python/lib/python3.10/xml/dom/NodeFilter.py system/etc/python/lib/python3.10/xml/dom/xmlbuilder.py system/etc/python/lib/python3.10/xml/dom/minicompat.py system/etc/python/lib/python3.10/xml/dom/expatbuilder.py system/etc/python/lib/python3.10/xml/dom/pulldom.py system/etc/python/lib/python3.10/xml/dom/minidom.py system/etc/python/lib/python3.10/xml/dom/__init__.py system/etc/python/lib/python3.10/xml/dom/domreg.py system/etc/python/lib/python3.10/xml/__init__.py system/etc/python/lib/python3.10/xml/etree system/etc/python/lib/python3.10/xml/etree/ElementInclude.py system/etc/python/lib/python3.10/xml/etree/ElementTree.py system/etc/python/lib/python3.10/xml/etree/cElementTree.py system/etc/python/lib/python3.10/xml/etree/__init__.py system/etc/python/lib/python3.10/xml/etree/ElementPath.py system/etc/python/lib/python3.10/xml/sax system/etc/python/lib/python3.10/xml/sax/_exceptions.py system/etc/python/lib/python3.10/xml/sax/saxutils.py system/etc/python/lib/python3.10/xml/sax/expatreader.py system/etc/python/lib/python3.10/xml/sax/xmlreader.py system/etc/python/lib/python3.10/xml/sax/handler.py system/etc/python/lib/python3.10/xml/sax/__init__.py system/etc/python/lib/python3.10/xml/parsers system/etc/python/lib/python3.10/xml/parsers/expat.py system/etc/python/lib/python3.10/xml/parsers/__init__.py system/etc/python/lib/python3.10/copyreg.py system/etc/python/lib/python3.10/base64.py system/etc/python/lib/python3.10/sysconfig.py system/etc/python/lib/python3.10/ntpath.py system/etc/python/lib/python3.10/weakref.py system/etc/python/lib/python3.10/reprlib.py system/etc/python/lib/python3.10/io.py system/etc/python/lib/python3.10/keyword.py system/etc/python/lib/python3.10/stat.py system/etc/python/lib/python3.10/_sitebuiltins.py system/etc/python/lib/python3.10/sre_compile.py system/etc/python/lib/python3.10/operator.py system/etc/python/lib/python3.10/re.py system/etc/python/lib/python3.10/site.py system/etc/python/lib/python3.10/encodings system/etc/python/lib/python3.10/encodings/cp775.py system/etc/python/lib/python3.10/encodings/iso8859_14.py system/etc/python/lib/python3.10/encodings/cp1251.py system/etc/python/lib/python3.10/encodings/cp855.py system/etc/python/lib/python3.10/encodings/cp720.py system/etc/python/lib/python3.10/encodings/mac_iceland.py system/etc/python/lib/python3.10/encodings/utf_8_sig.py system/etc/python/lib/python3.10/encodings/cp874.py system/etc/python/lib/python3.10/encodings/mac_romanian.py system/etc/python/lib/python3.10/encodings/uu_codec.py system/etc/python/lib/python3.10/encodings/idna.py system/etc/python/lib/python3.10/encodings/mac_roman.py system/etc/python/lib/python3.10/encodings/mac_arabic.py system/etc/python/lib/python3.10/encodings/rot_13.py system/etc/python/lib/python3.10/encodings/undefined.py system/etc/python/lib/python3.10/encodings/latin_1.py system/etc/python/lib/python3.10/encodings/cp424.py system/etc/python/lib/python3.10/encodings/iso8859_2.py system/etc/python/lib/python3.10/encodings/utf_32_be.py system/etc/python/lib/python3.10/encodings/cp1125.py system/etc/python/lib/python3.10/encodings/cp860.py system/etc/python/lib/python3.10/encodings/cp858.py system/etc/python/lib/python3.10/encodings/cp857.py system/etc/python/lib/python3.10/encodings/shift_jis_2004.py system/etc/python/lib/python3.10/encodings/cp1258.py system/etc/python/lib/python3.10/encodings/cp037.py system/etc/python/lib/python3.10/encodings/mac_greek.py system/etc/python/lib/python3.10/encodings/cp864.py system/etc/python/lib/python3.10/encodings/quopri_codec.py system/etc/python/lib/python3.10/encodings/iso2022_jp_2.py system/etc/python/lib/python3.10/encodings/ptcp154.py system/etc/python/lib/python3.10/encodings/gb2312.py system/etc/python/lib/python3.10/encodings/cp862.py system/etc/python/lib/python3.10/encodings/iso2022_jp.py system/etc/python/lib/python3.10/encodings/cp949.py system/etc/python/lib/python3.10/encodings/iso8859_1.py system/etc/python/lib/python3.10/encodings/iso8859_10.py system/etc/python/lib/python3.10/encodings/cp869.py system/etc/python/lib/python3.10/encodings/charmap.py system/etc/python/lib/python3.10/encodings/cp1257.py system/etc/python/lib/python3.10/encodings/cp866.py system/etc/python/lib/python3.10/encodings/big5hkscs.py system/etc/python/lib/python3.10/encodings/shift_jisx0213.py system/etc/python/lib/python3.10/encodings/cp1254.py system/etc/python/lib/python3.10/encodings/base64_codec.py system/etc/python/lib/python3.10/encodings/hz.py system/etc/python/lib/python3.10/encodings/cp1250.py system/etc/python/lib/python3.10/encodings/palmos.py system/etc/python/lib/python3.10/encodings/cp861.py system/etc/python/lib/python3.10/encodings/zlib_codec.py system/etc/python/lib/python3.10/encodings/iso2022_jp_1.py system/etc/python/lib/python3.10/encodings/gbk.py system/etc/python/lib/python3.10/encodings/johab.py system/etc/python/lib/python3.10/encodings/iso8859_11.py system/etc/python/lib/python3.10/encodings/tis_620.py system/etc/python/lib/python3.10/encodings/ascii.py system/etc/python/lib/python3.10/encodings/koi8_u.py system/etc/python/lib/python3.10/encodings/iso8859_8.py system/etc/python/lib/python3.10/encodings/iso2022_jp_ext.py system/etc/python/lib/python3.10/encodings/euc_jp.py system/etc/python/lib/python3.10/encodings/shift_jis.py system/etc/python/lib/python3.10/encodings/cp273.py system/etc/python/lib/python3.10/encodings/punycode.py system/etc/python/lib/python3.10/encodings/utf_7.py system/etc/python/lib/python3.10/encodings/iso2022_kr.py system/etc/python/lib/python3.10/encodings/cp1255.py system/etc/python/lib/python3.10/encodings/iso8859_9.py system/etc/python/lib/python3.10/encodings/iso2022_jp_3.py system/etc/python/lib/python3.10/encodings/koi8_t.py system/etc/python/lib/python3.10/encodings/utf_32.py system/etc/python/lib/python3.10/encodings/cp1026.py system/etc/python/lib/python3.10/encodings/mac_latin2.py system/etc/python/lib/python3.10/encodings/cp856.py system/etc/python/lib/python3.10/encodings/oem.py system/etc/python/lib/python3.10/encodings/mac_croatian.py system/etc/python/lib/python3.10/encodings/hp_roman8.py system/etc/python/lib/python3.10/encodings/__init__.py system/etc/python/lib/python3.10/encodings/cp863.py system/etc/python/lib/python3.10/encodings/koi8_r.py system/etc/python/lib/python3.10/encodings/iso8859_7.py system/etc/python/lib/python3.10/encodings/cp865.py system/etc/python/lib/python3.10/encodings/utf_16.py system/etc/python/lib/python3.10/encodings/cp1253.py system/etc/python/lib/python3.10/encodings/mac_turkish.py system/etc/python/lib/python3.10/encodings/cp1006.py system/etc/python/lib/python3.10/encodings/cp932.py system/etc/python/lib/python3.10/encodings/kz1048.py system/etc/python/lib/python3.10/encodings/cp437.py system/etc/python/lib/python3.10/encodings/raw_unicode_escape.py system/etc/python/lib/python3.10/encodings/utf_8.py system/etc/python/lib/python3.10/encodings/cp1252.py system/etc/python/lib/python3.10/encodings/big5.py system/etc/python/lib/python3.10/encodings/utf_16_be.py system/etc/python/lib/python3.10/encodings/euc_jisx0213.py system/etc/python/lib/python3.10/encodings/iso8859_5.py system/etc/python/lib/python3.10/encodings/mac_farsi.py system/etc/python/lib/python3.10/encodings/unicode_escape.py system/etc/python/lib/python3.10/encodings/cp950.py system/etc/python/lib/python3.10/encodings/euc_kr.py system/etc/python/lib/python3.10/encodings/cp500.py system/etc/python/lib/python3.10/encodings/gb18030.py system/etc/python/lib/python3.10/encodings/iso2022_jp_2004.py system/etc/python/lib/python3.10/encodings/cp1140.py system/etc/python/lib/python3.10/encodings/cp852.py system/etc/python/lib/python3.10/encodings/mac_cyrillic.py system/etc/python/lib/python3.10/encodings/hex_codec.py system/etc/python/lib/python3.10/encodings/iso8859_3.py system/etc/python/lib/python3.10/encodings/mbcs.py system/etc/python/lib/python3.10/encodings/bz2_codec.py system/etc/python/lib/python3.10/encodings/iso8859_16.py system/etc/python/lib/python3.10/encodings/utf_16_le.py system/etc/python/lib/python3.10/encodings/euc_jis_2004.py system/etc/python/lib/python3.10/encodings/iso8859_4.py system/etc/python/lib/python3.10/encodings/iso8859_15.py system/etc/python/lib/python3.10/encodings/iso8859_13.py system/etc/python/lib/python3.10/encodings/cp850.py system/etc/python/lib/python3.10/encodings/cp737.py system/etc/python/lib/python3.10/encodings/iso8859_6.py system/etc/python/lib/python3.10/encodings/cp1256.py system/etc/python/lib/python3.10/encodings/cp875.py system/etc/python/lib/python3.10/encodings/aliases.py system/etc/python/lib/python3.10/encodings/utf_32_le.py system/etc/python/lib/python3.10/site-packages system/lib system/lib/python3.10 system/lib/python3.10/site-packages system/lib/python3.10/lib-dynload system/lib/python3.10/lib-dynload/_struct.cpython-310.so system/lib/python3.10/lib-dynload/binascii.cpython-310.so ┌root@localhost:/data/adb/modules/python3 └─# site-packages config The Magisk Module contains two directories fo site-packages: `┌shell@localhost:/data/local/tmp/develop/work└─$ python -c 'import site; print(site.getsitepackages())'['/etc/python/lib/python3.10/site-packages', '/system/lib/python3.10/site-packages']┌shell@localhost:/data/local/tmp/develop/work` /system/lib/python3.10/site-packages can be used to add python packages to the Magisk Module. /etc/python/lib/python3.10/site-packages is a symbolic link to a directory writable by the user shell that can be used to add site packages to the already installed python: `┌root@localhost:/data/adb/modules/python3└─# ls -ld /etc/python/lib/python3.10/site-packageslrwxrwxrwx 1 root root 40 1970-03-16 23:40 /etc/python/lib/python3.10/site-packages -> /data/local/tmp/python3.10/site-packages┌root@localhost:/data/adb/modules/python3└─#`
python3_3.14.v1.0.0.zip	This Magisk Module contains Python3 3.14 for arm64 CPUs	see here for details
cmake_3.30.4.zip	This Magisk Module contains cmake binaries for arm64 CPUs	The Magisk Module installs the binaries in /system/bin. Files in the module `┌root@localhost:/data/adb/modules/cmake/system└─# date``Mon Sep 30 17:31:30 CEST 2024``┌root@localhost:/data/adb/modules/cmake/system└─# ls -ld $( find . -type f )` `-rwxr-xr-x 1 root root 193642208 2024-09-30 16:15 ./bin/cmake-rwxr-xr-x 1 root root 197898576 2024-09-30 16:15 ./bin/cpack-rwxr-xr-x 1 root root 214667032 2024-09-30 16:15 ./bin/ctest-rwxr-xr-x 1 root root 214190168 2024-09-30 16:15 ./bin/ctresalloc-rwxr-xr-x 1 root root 845712 2024-09-30 16:15 ./lib64/libuv.so┌root@localhost:/data/adb/modules/cmake/system` └─#
autoconf_tools_1.1.0.0.zip	This Magisk Module contains the autoconf util for Android; the executables for autoconf are all Perl scripts but the necessary m4 is a binary for arm64 CPUs.	The Magisk Module installs the binaries in /system/bin. The executables in this module require Perl (/system/bin/perl) Files in the module ┌┌root@localhost:/data/adb/modules/autoconf_tools/system/bin └─# date Thu Oct 3 12:07:19 CEST 2024 ┌root@localhost:/data/adb/modules/autoconf_tools/system/bin └─# ┌root@localhost:/data/adb/modules/autoconf_tools/system/bin └─# ls -l total 1244 lrwxrwxrwx 1 root root 14 2024-10-03 10:33 aclocal -> ./aclocal-1.17 -rwxr-xr-x 1 root root 37744 2024-10-03 10:33 aclocal-1.17 -rwxr-xr-x 1 root root 6708 2024-10-03 10:33 autoconf -rwxr-xr-x 1 root root 9111 2024-10-03 10:33 autoheader -rwxr-xr-x 1 root root 34943 2024-10-03 10:33 autom4te lrwxrwxrwx 1 root root 15 2024-10-03 10:33 automake -> ./automake-1.17 -rwxr-xr-x 1 root root 264364 2024-10-03 10:33 automake-1.17 -rwxr-xr-x 1 root root 26892 2024-10-03 10:33 autoreconf -rwxr-xr-x 1 root root 17196 2024-10-03 10:33 autoscan -rwxr-xr-x 1 root root 34039 2024-10-03 10:33 autoupdate -rwxr-xr-x 1 root root 4198 2024-10-03 10:33 ifnames -rwxr-xr-x 1 root root 373710 2024-10-03 10:33 libtool -rwxr-xr-x 1 root root 137168 2024-10-03 10:33 libtoolize -rwxr-xr-x 1 root root 297072 2024-10-03 10:33 m4 ┌root@localhost:/data/adb/modules/autoconf_tools/system/bin └─# ... and the necessary cata files The version of the tools in the Module are: autoconf 2.72 automake 1.17 m4 1.4.19 libtool 2.5.3 Notes To use an m4 executable in another directory set the environment variable M4 before calling autoconf.
bison_flex_1.0.0.0.zip	This Magisk Module contains bsion, yacc, and flex for arm64 CPUs .	The Magisk Module installs the binaries in /system/bin. The executables in this module require m4 (/system/bin/m4) Files in the module `┌root@localhost:/data/adb/modules/bison_flex/system/bin └─# date Wed Oct 2 12:17:52 CEST 2024 ┌root@localhost:/data/adb/modules/bison_flex/system/bin └─# root@localhost:/data/adb/modules/bison_flex/system/bin└─# ls -ltrtotal 1460lrwxrwxrwx 1 root root 4 2024-10-02 12:10 flex++ -> flex-rwxr-xr-x 1 root root 513936 2024-10-02 12:10 flex-rwxr-xr-x 1 root root 4221 2024-10-02 12:10 yacc-rwxr-xr-x 1 root root 969104 2024-10-02 12:10 bison┌root@localhost:/data/adb/modules/bison_flex/system/bin└─#` and the necessary data files The version of the tools in the Module are: bison 3.8 flex 2.6.4
openssh_9.9p1.v1.0.0.zip	This Magisk Module contains OpenSSH for arm64 CPUs	see here for details
sqlite3_3.47.0.v1.0.0.zip	This Magisk Module contains sqlite3 for arm64 CPUs	The Magisk Module installs the binaries in /system/bin. Files in the module `┌root@localhost:/data/adb/modules/sqlite3└─# dateMon Oct 7 13:55:22 CEST 2024┌root@localhost:/data/adb/modules/sqlite3└─#` `┌root@localhost:/data/adb/modules/sqlite3└─# find .../system./system/usr./system/usr/include./system/usr/include/sqlite3ext.h./system/usr/include/sqlite3.h./system/usr/bin./system/usr/bin/sqlite3./system/usr/lib./system/usr/lib/libsqlite3.la./system/usr/lib/pkgconfig./system/usr/lib/pkgconfig/sqlite3.pc./system/usr/lib/libsqlite3.a./system/bin./system/bin/sqlite3./source./uninstall.sh./magiskpolicy./module.prop./create_zip.sh./README./common./common/functions.sh┌root@localhost:/data/adb/modules/sqlite3└─#` ---- To only use the sqlite3 binary from the Module use this command: `unzip -p ./sqlite3_3.47.0.v1.0.0.zip system/usr/bin/sqlite3>sqlite3`

Note:

To change the content of one of the Magisk modules listed above, unzip the ZIP file, change/add/delete files and recreate the ZIP file with the Linux shell script create_zip.sh from the ZIP file.

Other Android related files

Show or Hide Table

File	Content	Comment
twrp-3.7.0_12-0-I006D-with_Android14_support.img	in-official (!) TWRP image for the ASUS Zenfone 8 with support for Android 14 Use at your own risk!	The source for this TWRP image was checked out on 25.11.2023. There are additional properties defined in this image: add. properties in the image `ASUS_I006D:/ # getprop \| grep android_version [ro.product.bootimage.supported_android_version]: [14] [ro.product.odm.supported_android_version]: [14] [ro.product.product.supported_android_version]: [14] [ro.product.system_ext.supported_android_version]: [14] [ro.product.vendor.supported_android_version]: [14] ASUS_I006D:/ #` Update 10.01.2024 There is now an official new TWRP version for ASUS Zenfone 8 with Android 14 support: TWRP version 3.7.0.12-1
twrp-3.7.0_12-0-I006D-with_Android14_support_and_tools.img	in-official (!) TWRP image for the ASUS Zenfone 8 with support for Android 14 and additional binaries Use at your own risk!	this is the TWRP image twrp-3.7.0_12-0-I006D-with_Android14_support.img with these additional binaries: list of add. binaries ASUS_I006D:/ # ls -l /system/bin/ed system/bin/gawk /system/bin/gdisk /system/bin/parted /system/bin/sqlite3 /system/bin/vi/system/xbin -rwxrwxrwx 1 root root 744128 2023-11-26 09:08 /system/bin/ed -rwxrwxrwx 1 root root 680612 2023-11-26 09:08 /system/bin/gdisk -rwxrwxrwx 1 root root 470788 2023-11-26 09:08 /system/bin/parted -rwxrwxrwx 1 root root 1714720 2023-11-26 09:08 /system/bin/sqlite3 -rwxrwxrwx 1 root root 4127280 2023-11-26 09:09 /system/bin/vi -rwxrwxrwx 1 root root 1777096 2023-11-26 09:08 system/bin/gawk /system/xbin: total 4500 -rwxrwxrwx 1 root root 938296 2023-11-26 09:06 cpio -rwxrwxrwx 1 root root 960496 2023-11-26 09:06 find -rwxrwxrwx 1 root root 1486120 2023-11-26 09:06 grep -rwxrwxrwx 1 root root 1216256 2023-11-26 09:06 tar ASUS_I006D:/ # This image also defines these two new properties: add. properties `ASUS_I006D:/ # getprop ro.product.type enhanced ASUS_I006D:/ # ASUS_I006D:/ # getprop ro.source_checkout_date 2023-11-25 ASUS_I006D:/ #`
twrp-3.7.0_12-1-I006D-enhanced.img	in-official (!) TWRP image for the ASUS Zenfone 8 with support for Android 14 Use at your own risk!	this is a copy of the official twrp-3.7.0_12-1-I006D.img with some additional files: list of add. binaries ASUS_I006D:/system/bin # pwd /system/bin ASUS_I006D:/system/bin # ls -l bootctl* lp* ed gawk gdisk parted sqlite3 vi* /system/xbin -rwxr-x--- 1 root root 34112 2024-01-10 19:54 bootctl -rwxr-x--- 1 root root 19832 2024-01-10 19:54 bootctl.bin -rwxr-xr-x 1 root root 744128 2024-01-10 19:54 ed -rwxr-xr-x 1 root root 1777096 2024-01-10 19:54 gawk -rwxr-xr-x 1 root root 680612 2024-01-10 19:54 gdisk -rwxr-xr-x 1 root root 19872 2024-01-10 19:54 lpdump -rwxr-xr-x 1 root root 31424 2024-01-10 19:54 lpdumpd -rwxr-xr-x 1 root root 100712 2024-01-10 19:54 lptools -rwxr-xr-x 1 root root 470788 2024-01-10 19:54 parted -rwxr-xr-x 1 root root 1714720 2024-01-10 19:54 sqlite3 -rwxr-xr-x 1 root root 4127280 2024-01-10 19:54 vi /system/xbin: total 4500 -rwxr-xr-x 1 root root 938296 2024-01-10 19:54 cpio -rwxr-xr-x 1 root root 960496 2024-01-10 19:54 find -rwxr-xr-x 1 root root 1486120 2024-01-10 19:54 grep -rwxr-xr-x 1 root root 1216256 2024-01-10 19:54 tar ASUS_I006D:/system/bin # see How to add additional files to an TWRP image for how to add additional files to a TWRP image

twrp_3.7.0_12-0-I006D_for_lineageOS20-2024.img	26.04.2024 Looks like this dirty created image only works for a specific LineageOS build. Therefor I removed the file. If you need this kind of TWRP image please create it yourself using the instructions from this post: How to use TWRP if LineageOS 20.x is installed (see also How to create a TWRP image for LineageOS based ROMs using a script)
twrp_3.7.1-12-1_ASUS_Zenfone8_2024-08-12_extended.img	in-official (!) TWRP image for the ASUS Zenfone 8 with additional binaries Use at your own risk!	The source for this TWRP image was checked out on 10.08.2024. Additional binaries and files in the image are: list of add. binaries and files ASUS_I006D:/ # date Mon Aug 12 09:46:56 CDT 2024 ASUS_I006D:/ # ASUS_I006D:/ # ls -ld $( tail -20 ramdisk-files.txt ) system/bin/lp* system/bin/boot* -rwxr-xr-x 1 root root 628616 2024-08-12 09:20 system/bin/bmore -rwxr-xr-x 1 root root 34112 1969-12-31 19:00 system/bin/bootctl -rwxr-xr-x 1 root root 824328 2024-08-12 09:20 system/bin/bvi -rwxr-xr-x 1 system system 744128 2024-08-10 01:30 system/bin/ed -rwxr-xr-x 1 system system 1777096 2024-08-10 01:30 system/bin/gawk -rwxr-xr-x 1 system system 680612 2024-08-10 01:30 system/bin/gdisk -rwxr-xr-x 1 root root 19872 1969-12-31 19:00 system/bin/lpdump -rwxr-xr-x 1 root root 31424 1969-12-31 19:00 system/bin/lpdumpd -rwxr-xr-x 1 root root 100712 1969-12-31 19:00 system/bin/lptools -rwxr-xr-x 1 system system 470788 2024-08-10 01:30 system/bin/parted -rwxr-xr-x 1 system system 1714720 2024-08-10 01:30 system/bin/sqlite3 lrwxrwxrwx 1 root root 5 2024-08-12 09:21 system/bin/vi -> ./vim lrwxrwxrwx 1 root root 5 2024-08-12 09:21 system/bin/vi -> ./vim lrwxrwxrwx 1 root root 5 2024-08-12 09:21 system/bin/view -> ./vim -rwxr-xr-x 1 root root 3666232 2024-08-12 09:20 system/bin/vim lrwxrwxrwx 1 root root 5 2024-08-12 09:21 system/bin/vimdiff -> ./vim -rw-r--r-- 1 root root 387592 2024-08-10 07:34 system/lib64/libncursesw.so.6 drwxrwxrwx 3 root root 60 2024-08-12 09:10 system/usr/share/vim drwxrwxrwx 2 root root 60 2024-08-12 09:10 system/usr/share/vim/vim90 -rw-rw-rw- 1 root root 0 2024-08-10 07:44 system/usr/share/vim/vim90/defaults.vim -rwxr-xr-x 1 system system 938296 2024-08-10 01:31 system/xbin/cpio -rwxr-xr-x 1 system system 960496 2024-08-10 01:31 system/xbin/find -rwxr-xr-x 1 system system 1486120 2024-08-10 01:31 system/xbin/grep -rwxr-xr-x 1 system system 1216256 2024-08-10 01:31 system/xbin/tar ASUS_I006D:/ # This image also defines these two new properties: additional properties `ASUS_I006D:/ # getprop ro.product.type enhanced ASUS_I006D:/ # ASUS_I006D:/ # getprop ro.source_checkout_date 2024-08-10 ASUS_I006D:/ #`
twrp_3.7.1-12-1_ASUS_Zenfone8_2024-09-10_extended.img	in-official (!) TWRP image for the ASUS Zenfone 8 with additional binaries Use at your own risk!	The source for this TWRP image was checked out on 10.09.2024. Additional binaries and files in the image are: list of add. binaries and files -rwxr-xr-x 1 root root 628616 2024-08-12 09:20 system/bin/bmore -rwxr-xr-x 1 root root 824328 2024-08-12 09:20 system/bin/bvi -rwxr-xr-x 1 root root 99160 2024-09-07 00:15 system/bin/dmctl -rwxr-xr-x 1 system system 744128 2024-08-10 01:30 system/bin/ed -rwxr-xr-x 1 system system 1777096 2024-08-10 01:30 system/bin/gawk -rwxr-xr-x 1 system system 680612 2024-08-10 01:30 system/bin/gdisk -rwxr-xr-x 1 root root 999064 2022-08-18 11:41 system/bin/lpadd -rwxr-xr-x 1 root root 1907128 2022-08-18 11:45 system/bin/lpdump -rwxr-xr-x 1 root root 969144 2022-08-18 11:45 system/bin/lpmake -rwxr-xr-x 1 root root 606704 2023-01-25 04:31 system/bin/make_ext4fs -rwxr-xr-x 1 system system 470788 2024-08-10 01:30 system/bin/parted -rwxr-xr-x 1 system system 1714720 2024-08-10 01:30 system/bin/sqlite3 lrwxrwxrwx 1 root root 5 2024-08-12 09:21 system/bin/vi -> ./vim lrwxrwxrwx 1 root root 5 2024-08-12 09:21 system/bin/vi -> ./vim lrwxrwxrwx 1 root root 5 2024-08-12 09:21 system/bin/view -> ./vim -rwxr-xr-x 1 root root 59 2024-09-10 10:52 system/bin/vim lrwxrwxrwx 1 root root 5 2024-08-12 09:21 system/bin/vimdiff -> ./vim -rw-r--r-- 1 root root 387592 2024-08-10 07:34 system/lib64/libncursesw.so.6 -rw-rw-rw- 1 root root 0 2024-08-10 07:44 system/usr/share/vim/vim91/defaults.vim -rwxr-xr-x 1 system system 938296 2024-08-10 01:31 system/xbin/cpio -rwxr-xr-x 1 system system 960496 2024-08-10 01:31 system/xbin/find -rwxr-xr-x 1 system system 1486120 2024-08-10 01:31 system/xbin/grep -rwxr-xr-x 1 system system 1216256 2024-08-10 01:31 system/xbin/tar This image also defines these two new properties: additional properties `qssi:/ # getprop ro.product.type enhanced qssi:/ # qssi:/ # getprop ro.source_checkout_date 2024-09-10 qssi:/ # qssi:/ # getprop ro.image_create_date 2024-09-10 qssi:/ #` Check https://gerrit.twrp.me/ for a list of changes in the source code
OrangeFox-Unofficial-I006D-2024-09-12.img OrangeFox-Unofficial-I006D-2024-09-12.zip	in-official (!) OrangeFox Recovery for the ASUS Zenfone 8 Use at your own risk!	The source for the OrangeFox recovery sources was checked out 10.09.204. The image was compiled with these custom build vars: Custom build vars `export FOX_REPLACE_BUSYBOX_PS=1export FOX_REPLACE_TOOLBOX_GETPROP=1export FOX_USE_TAR_BINARY=1export FOX_USE_SED_BINARY=1export FOX_USE_ZIP_BINARY=1export FOX_USE_NANO_EDITOR=1export FOX_USE_BASH_SHELL=1export OF_USE_MAGISKBOOT=1export OF_USE_MAGISKBOOT_FOR_ALL_PATCHES=1export FOX_DISABLE_APP_MANAGER=0export FOX_DELETE_AROMAFM=1export TARGET_DEVICE_ALT="gssi,ASUS_I006D"` seee https://gitlab.com/OrangeFox/vendor/recovery/-/blob/master/orangefox_build_vars.txt for a description of the custom build vars see https://wiki.orangefox.tech for infos about the OrangeFox recovery see How to compile the OrangeFox Recovery for the ASUS Zenfone 8 for instructions to create the image

General Android LInks

Show or Hide Table

URL	Content	Comment
https://xdaforums.com/	XDA Forum ; a very good source for Android phone related infos
https://microg.org/	MicroG homepage
https://nanolx.org/	NanoDroid homepage, NanoDroid is an installation image for MicroG
https://downloads.nanolx.org/NanoDroid/Stable/	NanoDroid Downloads
https://github.com/micro5k/microg-unofficial-installer/releases	Another installation image for MicroG

https://developer.android.com/studio/command-line/adb#shellcommands	adb shell command overview	Google account neccessary
https://developer.android.com/studio/command-line/adb#pm	usage help for the command "pm"	Google account neccessary
https://developer.android.com/studio/command-line/adb#am	usage help for the command "am"	Google account neccessary

https://developer.android.com/studio/releases/platform-tools	Android SDK Platform tools (fastboot, adb,etc)
https://developer.android.com/studio/releases/platform-tools.html	Android SDK Platform tools Downloads

https://github.com/roihershberg/bootctl-binary	bootctl binary to check or change the current slot
https://android.googlesource.com/platform/system/extras/+/master/bootctl/bootctl.cpp https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/boot_control.h	source code for bootctl

https://source.android.com/docs/core/ota/ab	A/B System updates documentation
https://dev.to/larsonzhong/most-complete-adb-commands-4pcg	ADB documentation

https://source.android.com/docs/core/architecture/bootloader/partitions	Partition usage in Android
https://developer.android.com/reference/android/provider/Settings.Global	Global system settings that can be changed with the command settings
https://developer.android.com/reference/android/provider/Settings.Secure	Secure system settings that can be changed with the command settings
https://developer.android.com/reference/android/provider/Settings.System	System settings, containing miscellaneous system preferences, that can be changed with the command settings
https://www.cclsolutionsgroup.com/post/android-abx-binary-xml	Description of the Android Binary XML files
https://source.android.com/docs/core/tests/debug/rescue-party	Android documentation about the "Android rescue trigger"

https://source.android.com/docs/security/bulletin	Android Security Bulletins
https://android.stackexchange.com/questions/219822/how-to-perform-a-hot-reboot-using-the-terminal-emulator	How to reboot Android
https://github.com/googlesamples/android-testdpc	"Test DPC is an app designed to help EMMs, ISVs, and OEMs to test their applications and platforms in a Android enterprise managed profile (i.e. work profile). It serves as both a sample Device Policy Controller and a testing application to flex the APIs available for Android enterprise. It supports devices running Android 5.0 Lollipop or later."	sample app from Google to test Android MDM (= Mobile Device Manager).
https://github.com/bkerler/edl	edl tool (python script)
https://www.temblast.com/edl.htm	edl tool for Windows
https://github.com/bkerler/Loaders	firehose loader for EDL
https://github.com/hoplik/Firehose-Finder/tree/master/fh_collection/	firehose loader for EDL
https://github.com/Alephgsm/SAMSUNG-EDL-Loaders/	firehose loader for EDL
http://www.temblast.com/ref/loaders.htm	a well done list of known firehose loader for EDL mode
https://alephsecurity.com/vulns/aleph-2017028	Indepth infos about the boot process for Android Devices with Qualcomm CPUs and EDL
https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/master/docs/user/adb.1.md	adb manpage
https://adbshell.com/	another adb commands manual

Useful Android apps and tools

Show or Hide Table

URL	App	Comment
https://f-droid.org/	F-Droid	"F-Droid is an installable catalogue of FOSS (Free and Open Source Software) applications for the Android platform. The client makes it easy to browse, install, and keep track of updates on your device"
https://github.com/x13a/Sentry?tab=readme-ov-file	Sentry	"Tiny app to enforce security policies of your device." This app can be used to disable the SafeMode
https://github.com/gibcheesepuffs/Switch-My-Slot-Android	Switch My Slot	app with a GUI for bootctl

https://hax4us.github.io/2021-11-22-install-android-sdk-in-termux/	Instructions to install the SDK in the Android OS
https://github.com/lzhiyong/termux-ndk	Android ndk for Termux (only supports aarch64 CPUs and Android 9 or above)
https://github.com/Lzhiyong/termux-ndk/releases/download/android-sdk/android-sdk-aarch64.zip	Android SDK compiled for aarch64 CPUs
https://github.com/lzhiyong/android-sdk-tools/releases/download/34.0.3/android-sdk-tools-static-aarch64.zip	binaries from the Android SDK for aarch64 CPUs There are also zip available files with the binaries for arm (32 Bit), i686, and x86_64 CPUs on this page Note that I created a Magisk Module with the binaries for aarch64 CPUs from this zip file -- see here.	Contents of the zip file [ OmniRom 14 Dev - xtrnaw7@t15g /data/img/android/aarch64 ] $ unzip -t android-sdk-tools-static-aarch64.zip Archive: android-sdk-tools-static-aarch64.zip testing: build-tools/aapt OK testing: build-tools/aapt2 OK testing: build-tools/aidl OK testing: build-tools/zipalign OK testing: build-tools/dexdump OK testing: build-tools/split-select OK testing: platform-tools/adb OK testing: platform-tools/fastboot OK testing: platform-tools/sqlite3 OK testing: platform-tools/dmtracedump OK testing: platform-tools/etc1tool OK testing: platform-tools/hprof-conv OK testing: platform-tools/e2fsdroid OK testing: platform-tools/sload_f2fs OK testing: platform-tools/mke2fs OK testing: platform-tools/make_f2fs OK testing: platform-tools/make_f2fs_casefold OK No errors detected in compressed data of android-sdk-tools-static-aarch64.zip. [ OmniRom 14 Dev - xtrnaw7@t15g /data/img/android/aarch64 ] $

TWRP related links

Show or Hide Table

URL	Content	Comment
https://twrp.me/	TWRP home page
https://twrp.me/faq/openrecoveryscript.html	TWRP command line guide

https://xdaforums.com/t/dev-how-to-compile-twrp-touch-recovery.1943625/	XDA Forum: How to compile TWRP touch recovery
https://www.hovatek.com/forum/thread-27922.html	How to enable and disable certain TWRP features without having kernel source code	not tested yet

https://github.com/TeamWin/Team-Win-Recovery-Project	TWRP source code	most probably outdated
https://github.com/TeamWin/android_bootable_recovery/	TWRP source code
https://github.com/TeamWin/android_bootable_recovery/blob/android-12.1/twrpRepacker.cpp	Source code from TWRP for installing TWRP into the boot partition
https://github.com/minimal-manifest-twrp/	Manifests for building TWRP
https://github.com/TeamWin/android_device_asus_I006D	TWRP device tree for ASUS ZenFone 8 a.k.a. "sake" a.k.a. ZS590KS
https://gerrit.twrp.me/q/status:open+-is:wip	Gerrit for TWRP

https://unofficialtwrp.com	Support for TWRP on devices offical not supported by TWRP
https://unofficialtwrp.com/build-compile-twrp-recovery/	Instructions how to build TWRP

https://github.com/Magisk-Modules-Repo/twrp-keep	A Magisk module with a script to reinstall TWRP into the inactive boot partition after installing an OS update

OrangeFox related Webpages	OrangeFox is a recovery image based on TWRP

https://wiki.orangefox.tech/	OrangeFox Home page
https://wiki.orangefox.tech/en/dev/building	Build instructions for OrangeFox
https://gitlab.com/OrangeFox/sync	Instructions to sync the OrangeFox repos

Magisk related links

Show or Hide Table

URL	Content	Comment
https://github.com/topjohnwu/Magisk	Magisk home page	Magisk the must have for every Android phone
https://topjohnwu.github.io/Magisk/install.html	Instructions to install Magisk
https://topjohnwu.github.io/Magisk/changes.html	Magisk changelog
https://topjohnwu.github.io/Magisk/	Magisk documentation
https://topjohnwu.github.io/Magisk/guides.html	Magisk developer guide
https://topjohnwu.github.io/Magisk/tools.html	Magisk Tools documentation
https://www.didgeridoohan.com/magisk/Magisk	Magisk TroubleShooting Guide
https://www.didgeridoohan.com/magisk/MagiskModuleIssues	Hints for issues with Magisk Modules
https://xdaforums.com/t/magisk-general-support-discussion.3432382/	General Magisk Support forum on XDA

https://github.com/DerGoogler/MMRL	Magisk Module Repo Loader -- a Magisk Module Manager	The MMRL requries this MagiskModule https://github.com/DerGoogler/MMRL-CLI
https://github.com/Fox2Code/FoxMagiskModuleManager	Fox Magisk ModuleManager
https://github.com/MRepoApp/magisk-modules-repo-util	A util for creating a Magisk Module repository

https://gist.github.com/niikoo/3f6bd13a69f2d68f3dd51cc667e79bdc	A startup script for Magisk to store the logcat messages
https://github.com/topjohnwu/zygisk-module-sample	zygisk sample module ; can be used start Zygisk modules
https://xdaforums.com/t/magisk-general-support-discussion.3432382/page-2776#post-88438781	post about the Evolution of the Android and Magisk booting process and architecture
https://infosecwriteups.com/adding-root-certificate-to-android-with-magisk-module-92493a7e9e4f?gi=fd1225db2b4a	Adding root certificates using a Magisk module

https://topjohnwu.github.io/Magisk/build.html	Instructions to build Magisk from source	Note: It's important to use `git clone --recurse-submodules https://github.com/topjohnwu/Magisk.git` to clone the repository see also the instructions for compiling Magisk here.
https://github.com/topjohnwu/ondk/releases	Android NDK tar files necessary to build Magisk	"Oxidized NDK (ONDK) is an unofficial repackaged Android NDK that includes a Rust toolchain."

Magisk Modules

Show or Hide Table

URL	Content	Comment

https://xdaforums.com/t/collection-of-magisk-modules-v2.3575758/	An XDA thread with a collection of Magisk Modules

https://github.com/Magisk-Modules-Repo/	Repository with Magisk Modules
https://github.com/Magisk-Modules-Alt-Repo/	Another repository with Magisk Modules
https://apt.izzysoft.de/magisk/	Repository with Magisk Modules
https://codeberg.org/fruitsnack/magisk-font-repo/raw/branch/main	Repository with Magisk Modules with fonts

https://github.com/mrh929/magisk-wifiadb	A Magisk module to automatically enable adb via WiFI
https://github.com/anasfanani/magisk-autoboot	"A Magisk module designed to enable automatic booting of your Android device when it's connected to a charger or USB."
https://github.com/anasfanani/Adb-Root-Enabler	A Magisk module to disable the authentication for the adb daemon
https://github.com/tiann/adb_root	A Magisk module to start the adbd as root user
https://github.com/Magisk-Modules-Alt-Repo/magisk_overlayfs	Magisk Module to create a writable overlay filesystem for the read-only filesystems
https://github.com/Magisk-Modules-Alt-Repo/NoProcStatRestriction	Magisk Module to allow access to /proc/stat for everyone again	This Magisk module can be used as example for how to apply a SELinux policy via Magisk Module
https://github.com/sn-00-x/fakestore2playstore/tree/master	A Magisk Module to replace the Fakestore with a patched Playstore

https://sourceforge.net/projects/magiskgapps/files/android-13-ALPHA/24.2.23/	Magisk Modules with Google Apps

https://github.com/Magisk-Modules-Repo/ssh	Magisk module with ssh and rsync
https://github.com/evdenis/disk	Disk tools for Android	`ASUS_I006D:/ # ls -l /data/adb/modules/disk/system/bin total 7924 -rwxr-xr-x 1 root root 1788544 2024-07-24 13:23 cgdisk -rwxr-xr-x 1 root root 547924 2024-07-24 13:23 fdisk -rwxr-xr-x 1 root root 1459440 2024-07-24 13:23 fixparts -rwxr-xr-x 1 root root 1566120 2024-07-24 13:23 gdisk -rwxr-xr-x 1 root shell 602784 2024-07-24 13:23 parted -rwxr-xr-x 1 root root 542056 2024-07-24 13:23 sfdisk -rwxr-xr-x 1 root shell 1591192 2024-07-24 13:23 sgdisk ASUS_I006D:/ #`
https://github.com/S-trace/tcpdump_static_aarch64	Magisk module with tcpdump for Android	version v4.9.2, static build
https://github.com/henriknelson/git-magisk-module	Magisk module with git for Android	version 2.30.2 The scripts from this package need /system/bin/bash
https://github.com/ianmacd/bash-aarch64	Magisk module with bash for Android	version 5.0.2
https://github.com/Magisk-Modules-Alt-Repo/mkshrc	Magisk module with mksh for Android
https://github.com/partcyborg/zsh_arm64_magisk	Magisk module with zsh for Android	The module does not work out of the box
https://github.com/henriknelson/opensshd-magisk-module	Magisk module with opensshd for Android	version v8.4p1
https://github.com/FerryAr/e2fsprogs-arm/releases/tag/v1.45.6	Magisk module with extfs tools for Android: `badblocks chattr dumpe2fs e2fsck e2image e2undo fsck lsattr mklost+found tune2fs base_device debugfs e2freefrag e2fuzz e2initrd_helper filefrag logsave mke2fs resize2fs`
https://apt.izzysoft.de/magisk/modules/SQLite3UniversalBinaries/1.5_15.zip Source: https://gitlab.com/adrian.m.miller/sqlite3universalbinaries	Magisk module with sqlite3 for Android	version 3.22
https://github.com/Magisk-Modules-Alt-Repo/sqlite3	Magisk module with sqlite3 for Android	version 3.46
https://github.com/Magisk-Modules-Alt-Repo/ToyBox-Ext	Magisk module with toybox with additional applets List of commands ASUS_I006D:/ $ su - -c /data/adb/modules/ToyBox-Ext/toybox-ext [ acpi arch ascii base32 base64 basename bash blkdiscard blkid blockdev bunzip2 bzcat cal cat chattr chgrp chmod chown chroot chrt chvt cksum clear cmp comm count cp cpio crc32 cut date dd deallocvt devmem df dirname dmesg dnsdomainname dos2unix du echo egrep eject env expand factor fallocate false fgrep file find flock fmt free freeramdisk fsfreeze fstype fsync ftpget ftpput getconf gpiodetect gpiofind gpioget gpioinfo gpioset grep groups gunzip halt head help hexedit host hostname httpd hwclock i2cdetect i2cdump i2cget i2cset i2ctransfer iconv id ifconfig inotifyd insmod install ionice iorenice iotop kill killall killall5 link linux32 ln logger login logname losetup ls lsattr lsmod lspci lsusb makedevs mcookie md5sum microcom mix mkdir mkfifo mknod mkpasswd mkswap mktemp modinfo mount mountpoint mv nbd-client nbd-server nc netcat netstat nice nl nohup nproc nsenter od oneit openvt partprobe passwd paste patch pgrep pidof ping ping6 pivot_root pkill pmap poweroff printenv printf prlimit ps pwd pwdx pwgen readahead readelf readlink realpath reboot renice reset rev rfkill rm rmdir rmmod route rtcwake sed seq setfattr setsid sh sha1sum sha224sum sha256sum sha384sum sha3sum sha512sum shred shuf sleep sntp sort split stat strings su swapoff swapon switch_root sync sysctl tac tail tar taskset tee test time timeout top touch toysh true truncate tty tunctl uclampset ulimit umount uname unicode uniq unix2dos unlink unshare uptime usleep uudecode uuencode uuidgen vconfig vmstat w watch watchdog wc wget which who whoami xargs xxd yes zcat ASUS_I006D:/ $
https://github.com/Magisk-Modules-Repo/ccbins https://github.com/Zackptg5/Cross-Compiled-Binaries-Android/tree/master	Cross Compiled binaries for Android
https://github.com/Zackptg5/Cross-Compiled-Binaries-Android	Source for the Magisk Module with various cross compiled binaries for Android	This zip file contains the binaries for different architectures [ OmniRomDev - xtrnaw7@t15g /data/develop/android/NewModules/Cross-Compiled-Binaries-Android-master ] $ ls -l bash/* -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 1541516 Sep 15 23:50 bash/bash-arm -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 1838864 Sep 15 23:50 bash/bash-arm64 -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 1963936 Sep 15 23:50 bash/bash-x64 -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 1883188 Sep 15 23:50 bash/bash-x86 -rwxr-xr-x. 1 xtrnaw7 xtrnaw7 3072 Sep 15 23:50 bash/etc.zip [ OmniRomDev - xtrnaw7@t15g /data/develop/android/NewModules/Cross-Compiled-Binaries-Android-master ] $ [ OmniRomDev - xtrnaw7@t15g /data/develop/android/NewModules/Cross-Compiled-Binaries-Android-master ] $ file bash/* bash/bash-arm: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, stripped bash/bash-arm64: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, stripped bash/bash-x64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped bash/bash-x86: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, stripped bash/etc.zip: Zip archive data, at least v2.0 to extract, compression method=store [ OmniRomDev - xtrnaw7@t15g /data/develop/android/NewModules/Cross-Compiled-Binaries-Android-master ] $
https://github.com/Magisk-Modules-Repo/ccbins	Magisk Module with a script to download various Unix tools compiled for Android from this repo: https://github.com/Zackptg5/Cross-Compiled-Binaries-Android	not compatible with Magisk v27+ according to the author
https://github.com/Googlers-Repo/gcc/	GCC toolchain for Android	As of 26.09.2024 the Magisk Module installs this gcc version: `┌shell@localhost:/data/local/tmp/develop/ncurses-6.3└─$ gcc --version` `aarch64-linux-android-gcc (GCC) 10.2.0Copyright (C) 2020 Free Software Foundation, Inc.This is free software; see the source for copying conditions. There is NOwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.┌shell@localhost:/data/local/tmp/develop/ncurses-6.3└─$` see Compiling C programs in Android using gcc for usage instructions for this module

Links for the OmniROM

Show or Hide Table

URL	Content	Comment
https://omnirom.org/	OmniROM Homage	OmniROM is Custom ROM for Android phones
https://xdaforums.com/c/omnirom.2601/	OmniROM Forums on XDA	Most of the forums are outdated; it seems that the forums are not used anymore

https://github.com/omnirom/android	Github repositories for OmniROM
https://gitlab.com/omnirom	Gitlab repositories for OmniROM
https://github.com/omnirom/android_prebuilts_prebuiltapks	List of MicroG packages in the OmniROM image with MicroG

https://github.com/omnirom/android_vendor_omni/blob/android-14.0/utils/aosp-forked-list	AOSP repositories forked for OmniROM	These repositories must be merged with the original AOSP repositories to apply the Android Security Updates to the OmniROM
https://github.com/omnirom/android_vendor_omni/blob/android-14.0/utils/aosp-merge.sh	Script to merge an Android Security Update to the local OmniROM repositories
https://www.inovex.de/de/blog/aosp-advanced-development-tricks/	ASOP : Advanced Development tricks
https://github.com/omnirom/android/tree/android-15	Github repositories for OmniROM 15

Links for MicroG

Show or Hide Table

URL	Content	Comment
https://microg.org/	MicroG Homepage
https://github.com/microg	MicroG source on Github
https://github.com/microg/GmsCore/wiki	MicroG Wiki
https://github.com/Psk-Ita/microG-GApps	Magisk module with MicroG with a patched Playstore	as of 08.07.2024 : the MicroG version is v0.2.29.233013 the Playstore version is 41.7.16-31 see How to install a more recent version of the Playstore in OmniROM
https://github.com/nift4/microg_installer_revived/	Magisk module with MicroG
https://github.com/micro5k/microg-unofficial-installer	"microG unofficial installer is a flashable zip created for a simple installation of microG on Android."	the support thread for this installer in XDA is available here: https://xdaforums.com/t/mod-flashable-microg-unofficial-installer.3432360/ Note: Version v1.3.1-beta successfully tested on plain OmniROM 14 without MicroG and GAPPS

Links for ROM Development

Show or Hide Table

URL	Content	Comment
https://gerrit.googlesource.com/git-repo/	repo script for buidling your own CustomROM
https://source.android.com/docs/setup/reference/repo	repo command reference
https://github.com/omnirom/android	Getting started doc for the OmniROM
https://github.com/omnirom	Repositories for building the OmniROM
https://developer.android.com/studio/command-line/	Documentation for the command line tools from the Android Studio
https://developer.android.com/studio/command-line/apksigner	Documentation for the apk signer from the Android studio
https://developer.android.com/studio/command-line/zipalign	Documentation for zipalign (zipalign must be used to align an APK file)	zipalign is part of the Android Studio
https://developer.android.com/studio	Android Studio Download	The link to download the Android Studio Command line tools is on the bottom of the page
https://developer.android.com/ndk/downloads	NDK Downloads
https://github.com/android/ndk/wiki/Unsupported-Downloads	NDK Downloads (old versions)
https://developer.android.com/tools#tools-sdk	Android Studio Command line tools

https://github.com/omnirom/android_prebuilts_prebuiltapks	Prebuild MicroG packages for the OmniROM
https://github.com/DonkeyCoyote/proprietary_vendor_asus	Vendor packages for the ASUS Zenfone 8 (neccessary for building your own CustomROM)
https://github.com/omnirom/android_device_asus_zenfone8/tree/android-12.1	Device tree for the ASUS Zenfone 8 (neccessary for building your own CustomROM)
https://github.com/TheMuppets/	Vendor packages for various devices
https://github.com/DonkeyCoyote/	Vendor packages for various devices
https://github.com/aosp-mirror/platform_build/tree/android12-platform-release/target/product/security	How to create a the ssl keys for a CustomROM image

https://source.android.com/docs/core/architecture/configuration/add-system-properties	How to define system properties
https://android.googlesource.com/platform/system/sepolicy/+/master/private/property_contexts	defined SELinux contexts

https://android.googlesource.com/platform/system/core/+/master/init/README.md	Android Init language
https://source.android.com/devices/tech/ota/sign_builds	Instructions to create certificates for an OS image

https://source.android.com/docs/security/bulletin	Android Security Bulletins
https://github.com/HyperN00B/Security_Patch_Merge_Helper	"This is a helper script made by me for Android Custom ROMs This script wil let you merge the selected security patch into the selected source code. This script only works with Custom ROM sources."
https://source.android.com/docs/setup/reference/build-numbers	Android code names, tags, and build numbers ("Patchlevel")
https://github.com/git-lfs/git-lfs	git command line extension lfs -- this is necessary to sync the repositories for OmniROM 14
https://github.com/falk-werner/zipsign	Tool to sign ZIP files using a SSL certificate	googletest (see below) is required to compile zipsign
https://github.com/google/googletest	googletest is required to compile zipsign
https://wiki.lineageos.org/verifying-builds	A tool verify the signature of a ZIP file with an ROM for an Android device
https://developer.android.com/ndk/guides/android_mk	Documentation for the syntax of Android.mk files
https://android.googlesource.com/platform/development/+/donut-release/ndk/docs/ANDROID-MK.TXT	Documentation for the syntax of Android.mk files

Links for useful Tools

Show or Hide Table

URL	Content	Comment
http://java-decompiler.github.io/#jd-gui-download	Java Decompiler	(jar file with GUI)
https://github.com/pxb1988/dex2jar/blob/2.x/README.md	Tools to work with android .dex and java .class files dex-reader/writer: Read/write the Dalvik Executable (.dex) file. It has a light weight API similar with ASM. d2j-dex2jar: Convert .dex file to .class files (zipped as jar) smali/baksmali: disassemble dex to smali files and assemble dex from smali files. different implementation to smali/baksmali, same syntax, but we support escape in type desc "Lcom/dex2jar\t\u1234;" other tools: d2j-decrypt-string
https://xdaforums.com/t/tool-imjtool-unpack-and-extract-a-variety-of-ota-images-from-various-vendors.4078159/ http://newandroidbook.com/tools/imjtool.html	mjtool - Unpack and extract a variety of OTA images from various vendors
https://xdaforums.com/t/tool-android-image-kitchen-unpack-repack-kernel-ramdisk-win-android-linux-mac.2073775/	Android Image Kitchen - Unpack/Repack Kernel Ramdisk [Win/Android/Linux/Mac]
https://github.com/hzw1199/xml2axml/releases	Convert compiled XML files used in apk files to plain XML files and vice versa
https://xdaforums.com/t/linux-guide-rom-payload-image-dump.4196055/	ROM payload image dumper - a python script to extract the images for the file payload.bin images for Android 12
https://github.com/ssut/payload-dumper-go	An android OTA payload dumper written in Go.
https://shatter-box.com/knowledgebase/android-apk-signing-tool-apk-signer/	Android APK Signing Tool (APK Signer)	Note; Android APK Signing Tool (APK Signer) Use the apk signer from the Android studio
https://ibotpeaches.github.io/Apktool/	A tool for reverse engineering Android apk files
https://xdaforums.com/t/tool-tickle-my-android-decompile-recompile-with-ease.1633333/	"TMA is a quick, powerful and easy-to-use tool that uses a piece of java called "apktool" to decompile and recompile Android app's. "
http://jsharkey.org/logcat/	python script that reformats the logcat output into a colorful stream that is much easier to visually follow.	this script is for Python 2.x

https://www.temblast.com/android.htm	Android applications, patches, and tools The binaries for the PC on this page are for Windows
https://github.com/radareorg/radare2	Disassembler with support for ARM CPUs
https://github.com/theopolis/uefi-firmware-parser	UEFI Firmware parser
https://github.com/Genymobile/scrcpy	"This application mirrors Android devices (video and audio) connected via USB or over TCP/IP, and allows to control the device with the keyboard and the mouse of the computer. It does not require any root access. It works on Linux, Windows and macOS."	very useful
https://github.com/GameTheory-/mktool	"mktool is for unpacking & repacking the android boot, recovery, or loki images and also for loki patching. Made for use on Linux. Must have java 8 or higher."
https://github.com/twrpdtgen/twrpdtgen	"Create a TWRP-compatible device tree only from an Android recovery image (or a boot image if the device uses non-dynamic partitions A/B) of your device's stock ROM It has been confirmed that this script supports images built starting from Android 4.4 up to Android 12"
https://github.com/sebaubuntu-python/aospdtgen	"Create a LineageOS-compatible device tree from an Android stock ROM dump (made with dumpyara). This script supports any Android firmware from a Treble-enabled device (Higher than Android 8.0 and with VNDK enabled, you can check it with Treble Info or with adb shell getprop ro.treble.enabled). For pre-Treble devices please use twrpdtgen."
https://github.com/sebaubuntu-python/dumpyara	dumpyara : "Android firmware dumper" - a tool to extract files from ROM images for trebled-enabled devices.
https://github.com/XayahSuSuSu/Android-DataBackup https://f-droid.org/zh_Hans/packages/com.xayah.databackup.foss/	Backup tool for Android (needs root access)

https://github.com/Heydarchi/SELinux-Explorer	SELinux explorer (python script for the PC):
https://github.com/xmikos/setools-android	setools ported to Android: - I created a Magisk module with these binaries - see the list of Magisk modules below
https://github.com/OpenDarwin-CVS/SEDarwin/blob/master/sedarwin7/src/sedarwin/policycoreutils/audit2allow/audit2allow.perl	Perl script to create SELinux tools -- can be used instead of the Linux executable audit2allow.

Links for the ASUS Zenfone 8

Show or Hide Table

URL	Content	Comment
https://www.asus.com/de/Mobile/Phones/ZenFone/Zenfone-8/HelpDesk_BIOS/	ASUS Zenfone 8 Firmware Download	As of 17.08.2023 ASUS removed all image files with OS images for the Zenfone 8 from their website
https://www.firmware27.info/2023/03/asus-zenfone-8-zs590ks-flash-file.html	inofficial page to download ASUS Zenfone 8 Firmware	17.08.2023 Most of the download links on this page are links to the original firmware page from ASUS and do not work anymore. The last two links are links to other webserver and still work as of 17.08.2023. See the section ASUS Zenfone 8 FIrmware Download with my comments regarding the image files from this page.
https://xdaforums.com/t/zenfone-8-firmware-ota-collection.4620171/	XDA Forum for ASUS Zenfone 8 Firmware Download Collection

https://twrp.me/asus/zenfone8.html	official TWRP for the ASUS Zenfone 8	Note that as of 09.01.2024 the official TWRP 3.7.0_12 for the ASUS Zenfone 8 does not support mouting the /data partition from Android 14 based OSse
https://build.twrp.me/twrp-3.7.0_12-1_TEST-I006D.img	TWRP for the ASUS Zenfone 8 with support for mounting /data in Android 14 based OSes	link checked at 09.01.2024; successfully mounted in the /data partition of Omnirom 14 booting this image

https://zentalk.asus.com/en/discussion/27136/unlock-relock-bootloader-unofficially-for-zenfone-3	Instructions to unlock the ASUS Zenfone 8
https://xdaforums.com/f/asus-zenfone-8.12291/	XDA Forums for the ASUS Zenfone 8
https://xdaforums.com/t/recovery-twrp-for-zenfone-8.4329139/	XDA Forum for TWRP for the ASUS Zenfone 8
https://xdaforums.com/t/full-recover-to-stock-if-things-went-really-bad.4337467/	Instructions to install the Android raw image from ASUS on the ASUS Zenfone 8
https://www.asus.com/Content/Android-13-Beta/	Android 13 Beta and a raw image for Android 12 The image for Android 13 Beta is also a raw image.	As of 17.08.2023 ASUS removed all image files with OS images for the Zenfone 8 from their website
https://www.asus.com/Content/Android-12-Beta/	Android 12 Beta and a raw image for Android 11	As of 17.08.2023 ASUS removed all image files with OS images for the Zenfone 8 from their website
https://dl.omnirom.org/zenfone8/	OmniROM Images for the ASUS Zenfone 8
https://xdaforums.com/t/recovery-twrp-for-zenfone-8.4329139/	XDA Support thread for TWRP for ASUS Zenfone 8
https://gist.github.com/shakalaca/bbc439916f389a7275083f90539059de	fastboot oem comands for the ASUS ROG Phone II Most of the also work for the ASUS Zenfone 8
https://xdaforums.com/t/cant-unlock-bootloader-failed-to-unlock-your-device-please-try-again-later-24803.4586429/	XDA Thread about unlocking ASUS phones without using the ASUS unlock tool	As of 23.09.2023 there is not yet a solution to unlock an ASUS phone without the ASUS unlock tool
https://www.pentestpartners.com/security-blog/breaking-the-android-bootloader-on-the-qualcomm-snapdragon-660/	Breaking the Android Bootloader on the Qualcomm Snapdragon 660
https://worthdoingbadly.com/qcomxbl/	Comparing Qualcomm's XBL UEFI bootloaders on Snapdragon 820, 835, and 845	A blog entry about the bootloader for Android devices with Qualcomm CPUs

https://xdaforums.com/t/unofficial-bootloader-unlocking-2024-01-06-without-asus-server-q3-2023-muhahahaha.4649465/#post-89260224 https://xdaforums.com/t/unlocking-the-bootloader-and-rooting.4649239/post-89258039 https://mitmproxy.org/	XDA threads about how to unlock an ASUS phone now that the original apk and webserver used to official unlock the phone is not available anymore

Distributions for the ASUS Zenfone 8

Show or Hide Table

URL	Content	Comment
https://omnirom.org/	OmniROM
https://lineageos.org/	LineageOS Homepage
https://lineage.microg.org/	LineageOS with MicroG Homepage
https://download.lineageos.org/devices/sake/builds	Images and Recoveries for LineageOS	LineageOS is a Custom ROM for Android ; the Lineage Recovery image can be used to install the OmniROM on an Android phone
https://wiki.lineageos.org/devices/sake/install?	Instructions to install the Recovery from the LineageOS
https://doc.e.foundation/ https://doc.e.foundation/devices/sake	/e/	sake is a code name for the ASUS Zenfone 8
https://xdaforums.com/t/rom-preview-sake-13-statixos-v6-0.4500497/	StatiXOS	development stopped in July 2024; last release Android 14 with patchlevel 07/2024 -- see https://xdaforums.com/t/closed-rom-upsidedowncake-sake-14-statixos-v7-10.4500497/#post-87515011
https://github.com/StatiXOS/android_manifest	Manifests and instructions to build the StatixOS
https://github.com/StatiXOS/android_manifest
https://download.lineage.microg.org/sake/	LineageOS with MircoG for the ASUS Zenfone 8
https://github.com/mikooomich/android_device_asus_sake/releases	un-official LineageOS 21.x Images for the ASUS Zenfone 8
https://libremobileos.com/lmodroid	LMODroid
https://arrowos.net/	ArrowOS	support is discontinued; latest release is Android 11 based from 01/2022

Misc Links

Show or Hide Table

URL	Content	Comment
https://android.googlesource.com/kernel/msm/+/android-7.1.0_r0.2/Documentation/blockdev/zram.txt	ram: Compressed RAM based block devices Description of the zram device implemented in the Android OS
https://www.kernel.org/doc/html/latest/admin-guide/blockdev/zram.html	Description for zram devices from the Linux Kernel doc

https://android.stackexchange.com/questions/213353/how-to-run-an-executable-on-boot-and-keep-it-running/213623#213623	How to run an executable and keep it running
https://android.stackexchange.com/questions/214839/how-to-run-an-android-init-service-with-superuser-selinux-context	How to run an Android init service with superuser SELinux context?
https://harrisonsand.com/posts/patching-adb-root/	Patching the adb daemon to run as root	only necessary if Magisk or a similar tool allow root access is not usable
https://xdaforums.com/t/new-sqlite3-binary-v3-40-0-for-all-devices.4273049/	SQLite3 binaries for Android phones
https://www.sqlitetutorial.net/	A tutorial for using sqlite
https://sqlite.org/index.html	SQLite homepage
https://security.stackexchange.com/questions/196230/connection-between-pin-password-and-encryption-keys-in-android	Description how the file based encryption is implemented in Android
https://labs.withsecure.com/publications/how-secure-is-your-android-keystore-authentication	Details about Android Keystore Authentication implementations
https://blog.quarkslab.com/android-data-encryption-in-depth.html	Blog entry about Android data encryption
http://www.javadecompilers.com/	Online Java Decompiler
https://github.com/M0Rf30/android-udev-rules	Repository with udev rules for using adb
https://askubuntu.com/questions/645/how-do-you-reset-a-usb-device-from-the-command-line	source code for a little C program to reset an USB port
https://www.programmersought.com/article/962810691772/	dcmtl usage examples
https://github.com/SELinuxProject/selinux/wiki/Tools	general list of available tools for maintaing SELinux
https://github.com/termux/termux-packages/tree/master/packages	Source code for the packages in Termux (incliuding build scripts for the tools and libraries)

back top top