Showing posts with label Linux. Show all posts
Showing posts with label Linux. Show all posts

Sunday, 23 June 2019

The Motorola DSP56L307EVM signal processing board

Some information to get a Motorola dsp56L307evm digital signal processing evaluation board from 2001 working on a linux computer in 2019.

The dsp56002evm (top) and dsp56l307evm (bottom)
This is a kind of a next step after getting my old dsp56002evm working under linux.

Introduction


The dsp56L307 is a 24bit (dsp) processor and can run at 150MHz. Together with the Enhanced Filter Coprocessor (efcop) it can generate 270MIPS. I will need to do some real benchmarking, but it can probably be compared with something like a STM32F407 (arm-M4 core at 168MHz) for the calculation speed.However the 24bit Harvard architecture with 3 memory spaces makes it still different and not fully comparable with an arm-M4, which is probably much better in quite some tasks..

I have bought my board from "littlediode" on ebay. While not advertised as such, it did came as the full original EVM kit in original box; that is with documentation, software, power adapter and cable.


The first challenge is to find, roughly 15 years after discontinuation and changing the company from Motorola to Freescale to NXP, all the documentation and tools back in the archives. My board did come with a CD with software like assembler and simulator, but that was not the latest version. The box did not include DSP specific test software for the board.

The wayback machine, does help a bit to find old pages, manuals(pdf) and software back. There are Motorola related pages and Freescale related pages that can be found back. It is advisable to browse a bit between the different snapshots to try to try to find what you are looking for. Another source are old university pages; this dsp was used for some courses.

 Then are all the tools that can be found only for dos / windows available, which gives inherently inconveniences for using those tools on Linux.


Most of the dos/windows tools can be used under a wine console, this shares at least the filesystem. For the dsp56002EVM was it partly needed to use dosbox / dosemu to get the debugger working, which is way more cumbersome in usage.

My current status is:
  • I can use the official (windows) tools and IDE for writing and compiling code.
  • I can use openocd to upload the code and inspect memory and registers to validate the working. (It is not real debuging, for that I would need gdb)
The next sections give a bit more information for this.

Setting up the (Windows) toolchain

The latest version of the required tools are part of the Freescale Symphony studio. This package contains the assembler, linker, c-compiler and tools to change the COFF files in a more universal format.
To get the eclipse based IDE working is it necessary to install an old  Java runtime. I was able to get one from oracle. This does need an account / login.

After installing this with "wine" then we get three sub directories in the install directory:

├── [4.0K]  dsp56720-devtools 
│   ├── [4.0K]  dist
│   │   ├── [4.0K]  dsp56720
│   │   ├── [4.0K]  gcc
│   │   │   ├── [4.0K]  bin  <-- directory with all the tools !
│   │   │   ├── [4.0K]  dsp
│   │   │   ├── [4.0K]  etc
│   │   │   └── [4.0K]  lib
│   │   ├── [4.0K]  gdb
│   │   └── [4.0K]  openocd
│   │       ├── [4.0K]  bin
│   │       └── [4.0K]  driver
│   ├── [4.0K]  doc
│   └── [4.0K]  licenses
├── [4.0K]  eclipse <-- directory with "startsymphony.bat" to start eclips based IDE
│   ├── [4.0K]  configuration
│   ├── [4.0K]  features
│   ├── [ 12K]  plugins
│   └── [4.0K]  readme
└── [4.0K]  sample-projects
    ├── [4.0K]  ASM-Tutorial
    │   └── [4.0K]  Debug
    └── [4.0K]  C-tutorial
        └── [4.0K]  Debug
The batch file to start the eclips based IDE is setting some paths which are necessary for the tool chain to work correctly.  These paths have to be set as well when we want to use the tools, for instance the gcc based c compiler, outside of the IDE. The contents of the batch file is:

~/.wine/drive_c/Symphony-Studio/eclipse$ cat startsymphony.bat 
@echo off

set DDT_HOME=C:\Symphony-Studio\dsp56720-devtools
set G563_EXEC_PREFIX=%DDT_HOME%\dist\gcc\lib\
set PATH=%DDT_HOME%\dist\gcc\bin\;%PATH%
set DSPLOC=%DDT_HOME%\dist\gcc

start C:\Symphony-Studio\eclipse\symphony-studio.exe

exit

The Symphony-studio program can also be started from the start menu. It does look to have sometimes problems to start properly; the windows stay empty. A bit clicking and loading some files seem to help.
The Symphony-studio IDE with assembler output.
With some well chosen command line options is it also possible to get the C-compiler working from the wine console.

$ wineconsole
C:\Symphony-Studio\dsp56720-devtools\dist\gcc\bin>g563c -BC:\Symphony-Studio\dsp56720-devtools\dist\gcc\lib hola.c -o hola.cld

The assembler can also be used stand-alone:

$ wineconsole
C:\Symphony-Studio\dsp56720-devtools\dist\gcc\bin>asm56300 -A -lpass.lst -B pass.asm

Setting up OpenOCD

The current OpenOCD (version 0.10.0) as some issues and is not fully supporting the dsp563xx anymore. The register structure is asking for an "exist" parameter which was missing. And the read and write functions for the memory have an incorrect word size definition. I have submitted two patches to openocd to solve this. The patches can also be found here.  Depending when this is read can this be already included in the latest openocd version.

Further more is a configuration file for openocd needed to make the jtag interface, target and board work together.  This  is still work in progress. The contents for the target is for now as follows:

$ cat dsp5gl307.cfg
if { [info exists CHIPNAME] } { 
   set _CHIPNAME $CHIPNAME
} else {
   set _CHIPNAME dsp56L307
}

if { [info exists ENDIAN] } { 
   set _ENDIAN $ENDIAN
} else {
  # this defaults to a big endian
   set _ENDIAN big
}

if { [info exists CPUTAPID] } {
   set _CPUTAPID $CPUTAPID
} else {
   set _CPUTAPID 0x0181101d
}
#jtag scan chain
jtag newtap $_CHIPNAME cpu -irlen 4 -ircapture 1 -irmask 0x03 -expected-id $_CPUTAPID
#target configuration
set _TARGETNAME $_CHIPNAME.cpu
target create $_TARGETNAME dsp563xx -endian $_ENDIAN -chain-position $_TARGETNAME
#working area at base of ram
$_TARGETNAME configure -work-area-virt 0

And for the EVM board I have made the following configuration file:
$ cat dsp56l307evm.cfg
# Script for freescale DSP56L307EVM
#
# the build in wiggler Jtag interface
source [find interface/parport.cfg]
#jtag speed
adapter_khz 100
reset_config trst_and_srst
# Some additional delays to improve the initialisation.
adapter_nsrst_delay 100
jtag_ntrst_delay   20

source dsp56l307.cfg

#setup flash 
#missing for now; did not work yet

proc enable_fast_clock {} {
    # Programm a factor of 5 into the PLL 
    # 12.288MHz * 3 = 36.8MHz
    mwwx 0xFFFFFD 0x060002
    # enable a timer to see a LED flashing
    mwwx 0xFFFF8F 0x000021
}
# initialize only board specifics - reset, clock, adapter frequency
proc init_board {} {
    global _TARGETNAME
    
    $_TARGETNAME configure -event reset-start {
        # Removed the system reset to be able to have the platform run
        reset_config trst_only
        adapter_khz 500
    }

    $_TARGETNAME configure -event reset-init {
        # this will be done only once!
        # perform a tap and system reset to be sure that the memory is cleared
        reset_config trst_and_srst
        enable_fast_clock
        adapter_khz 500
    }
}

I will upload the final version (when the flash is also working) to git for inclusion into the openocd source.

When starting and working with openocd you should get something as on the pictures below:
Starting openocd with the dsp56l307evm.cfg config file
Connecting with telnet to the debug session

The dsp653xx target of openocd uses a special parameter "memspace" in which of the 4 memory spaces the data is put:
Set memspace #
  • 1 gives access to X data memory  (MEM_X = 0)
  • 2 gives acces to Y data memory (MEM_Y = 1)
  • 3 gives access to the combined X and Y memory called L memory (MEM_L = 3).
  • all other values result in access to P memory or program memory (MEM_P = 2)
This can be used to load data in all memory spaces as follows:

> set memspace 0  # set P memory as default
> load_image program.p 0x0 s19

> set memspace 1   # set X memory as default
> load_image program.x 0x0 s19

> set memspace 2   # set Y memory as default
> load_image program.y 0x0 s19


Checking if open ocd is working by programming one of the timers:
> mwwx 0xFFFF8F 0x21
> mdwx 0xFFFF8F

To change the PLL we can do this:

> mwwx 0xFFFFFD 0x060002    <--  PLL (2+1)=3 x 12.288MHz = 36.864MHz
> mwwx 0xFFFFFD 0x060008    <--  PLL (8+1) =9 x 12.288MHz = 110.592MHz
> mwwx 0xFFFFFD 0x06000B    <--  PLL (11+1) =12 x 12.288MHz = 147.456MHz

Which makes the timer go faster as well.

SREC

The (windows) SREC tool can be used to convert the special CLD COFF files to something more universal that can be loaded by openocd to program the dsp.
This tool has to be run with the following options:
$ srec -r -t 4 infile.cld 
The two necessary options are:
  • The -r option is needed to flip the direction of the words to fix the Endianness difference between host and the dsp. 
  • With the -t 4 option are the words expanded from 24 bit to 32bit.
this image can be loaded by openocd onto a target:
> load_image infile.p 0x0 s19

This srec tool can be called from the Symphony IDE to convert the files after compiling.

Saturday, 3 November 2018

Firewire audio interfaces (hardware)



After my slightly disappointing experience with the Soundblaster Audigy 2 ZS I decided to step up a bit with my "project". In the past was or maybe still is an external firewire audio interface considered the best performance. The clock for audio stream is supposedly more stable than with USB and external is there less noise than inside of computer case. Therefore are all the external devices considered semi professional and come with at least 24bit and 96kHz sample rates in combination with balanced and unbalanced signal inputs and outputs to reduce noise.

The support for Firewire devices under linux is a bit sketchy:
  1. As usual are manufactures of such devices not releasing drivers and or documentation to make (open source) drivers.
  2. From 2007 and on went firewire the way of the dodo in favorite of USB; less computers got firewire buses (Apple stopped supporting it) and more equipment (audio, camera etc.) switched to USB as interface.
  3. The ffado project is providing out of kernel drivers.
  4. The Alsa sound system started to supports also a few firewire interfaces.

The second point makes that it is now a days not so "meaningful" anymore to spend a lot of time in making new or improved drivers for Firewire equipment. At the other hand this makes that it it is possible to buy fairly cheaply old firewire equipment.

I looked a bit on the ffado supported hardware and I found the following three interfaces relative cheaply (<<50 euro) on ebay.
  • Terratec Producer Phase 24 FW
  • Edirol FA-66
  • M-Audio Firewire 410

The front and back of the three boxes are visible in the following picture:
Three (semi professional) Firewire audio interfaces

The following table gives an overview of the analogue interface functionality of the three boxes:
Brand Type Input Output
Mic Line Bits Sample rate Headphone Line Bits Sample rate
Terratec Producer Phase 24 0 2 24 192kHz 2 2 24 192kHz
Edirol (Roland) FA-66 2 2 24 192kHz [2] 4 24 192kHz
M-audio Firewire 410 2 [2] 24 96kHz 2x2 8 24 96kHz

Some notes to the above shown table:
  • The signal level of the headphone output of the Terratec Producer Phase 24 can be changed to line level which results in 4 line outputs.
  • The headphone output of the Edirol FA-66 interface is not independent but always connected with the first two analogue line outputs.
  • The M-audo Firewire 410 interface has not really 4 inputs it can use the 2 microphone inputs or the two line inputs at the back but not both at the same time.   And the two headphone outputs have independent volume control but have both the same signal.
The microphone inputs have the option to enable a 48V phantom power supply for microphones. The have a signal gain control and/or limiter. Both balanced (differential) and unbalanced (single ended) signals can be used as XLR connector or 1/4" jack plug.
At the digital side are there also some differences between the three boxes:
Brand Type Input Output
Midi SPDIF Optical MIDI SPDIF Optical
Terratec Producer Phase 24 [1] [1] 0 [1] [1] 0
Edirol (Roland) FA-66 1 0 1 1 0 1
M-audio Firewire 410 1 1 1 1 1 1
/>
Some notes to the above shown table:
  • For the Phase 24 interface box are the signals available on the 9 pin sub-d connector. Unfortunately I did not get this breakout cable, but I could find the connections by some measuring. See this blog post for more information.
  • The SPDIF and Optical input of the Firewire 410 cannot be used at the same time. They both are used as one data stream.
  • The Firewire 410 box can use ADAT signals on the digital input which would result in 8 channels more. However I have not tried this yet.
The difference between the three audio interfaces is also very much visible when looked at the mixer possibilities. The next figure shows the three "mixer" views of ffado-mixer.
The ffado mixer interface for the Phase 24 (left), Edirol FA66 (middle) and Firewire 410 (right).
  The Phase 24 interface offers a simple mixer for the 2 inputs and the 3 output signals. This mixer can be mapped to one of the output channels. Furthermore is it possible to select the signal level of the headphone output (line or headphone), select the synchronisation source (internal or spdif) and a kind of 5 level signal gain on the analogue input.
The FA66 interface has an even simpler interface; all the options have buttons at the box itself and cannot configured over software. Only the mixer for the 4 analogue and spdif signal is left in software.
The Firewire 410 interface has the most complex mixing function to match the 2 analogue input, 2 digital input and 8 audio stream signals to the 8 analogue, 2 digital and 2 headphone output signals.
This makes it clear that the M-audio Firewire 410 box is the more "professional" one of the three interfaces. It is a pity that this specific box has only 2 analogue inputs, other, more expensive, versions have more analogue inputs but are in general not well supported by the linux software.
However with firewire and ffado is this not a real issue; as long as the interfaces are on the same bus and the clock is synchronised is it possible to stitch them together as one big audio interface. This give the possibility to sample 8 analogue inputs at the same time. More about this in the software part.

Wednesday, 31 October 2018

Soundblaster Audigy 2 ZS and Linux

I started recently to look a bit around for a good audio card to get to a "studio" quality recordings under Linux. I didn't want to invest much money so I decided to buy something on ebay.

A short investigation pointed me in the direction of the Audigy 2 ZS sound card. A Creative product with interesting specifications:
  • 24bit 96kHz sampling (in and out)
  • 108dB signal noise ratio
  • EMU10K1/2 processor
  • Hardware wavetable synthesizer (4x 16 channel polyphony)
  • Firewire interface (which became the best feature of this card)
And that all in an attractive package for only 10.50 Euro:
A2zs002.jpg
Creative SoundBlaster Audigy 2 ZS
By The original uploader was Swaaye at English Wikipedia. - Transferred from en.wikipedia to Commons by Lockal using CommonsHelper., CC BY-SA 3.0, Link

The mixer options look great:
Alsamixer for Audigy 2 ZS

Reality


While a quick scan suggested that this card is supported under linux is the reality rather disappointing.  The Alsa page gives some information.

It looks that this card is in practice only supported for  44.1kHz or 48kHz sample rate. And the playback and capturing is mainly 16bits with some support for 24bits. This is an alsa limitation and in that sense is the card not really an upgrade from the "crapy" audio of a normal main-board.

This forces me to look a  further for a new solution that does deliver 24bit at a high 96kHz or 192kHz sample rate.

Wavetable synthesizer


This part has some hardware limitations.  According to alsa information page is the address bus only 31bits, which gives problems with a 64bit operating system and more than 4 Gigabyte memory. When this card was released was this probably a none issue, but now with only 64bit Operating systems and 16Gbyte of memory is this a bit of pity.
A work arround is saving some bigger memory block by using the following kernel parameter within the grub2 boot menu:
memmap=2048M\$6144M
It can be unfortunately necessary to play a bit with single (\), double (\\) or triple (\\\) to escape the $ sign within grub2 config file to make this work correctly automatically on boot time.
Furthermore can it be necessary to increase the normal 128MByte memory for the wave table to 200MByte to make the FluidR3_GM.sf2 sound font fit. This can be done by using the following kernel module parameter:
options snd-emu10k1 max_buffer_size=<size_in_MB>
This parameter can be put in the /etc/modprobe.d/alsa-base.conf file.

If this is correctly working can be checked with the following commands:
hansan@Desk-computer:~/Music/mid$ cat /proc/asound/cards
0 [FW             ]: BeBoB - PHASE 24 FW
                      TerraTec Electronic Gmb PHASE 24 FW (id:4, rev:1), GUID 000aac0400239b18 at fw2 
1 [PCH            ]: HDA-Intel - HDA Intel PCH
                      HDA Intel PCH at 0xef410000 irq 30 
2 [HDMI           ]: HDA-Intel - HDA ATI HDMI
                      HDA ATI HDMI at 0xef360000 irq 31 
3 [Audigy2        ]: Audigy2 - SB Audigy 2 ZS [SB0350]
                      SB Audigy 2 ZS [SB0350] (rev.4, serial:0x20021102) at 0xc000, irq 16 
4 [U0x46d0x9a2    ]: USB-Audio - USB Device 0x46d:0x9a2
                      USB Device 0x46d:0x9a2 at usb-0000:00:14.0-3, high speed
And to check the status of the synthesizer:
hansan@Desk-computer:~/Music/mid$ cat /proc/asound/card3/wavetableD1
Device: Emu10k1
Ports: 4
Addresses: 29:0 29:1 29:2 29:3
Use Counter: 0
Max Voices: 64
Allocated Voices: 0
Memory Size: 134217728
Memory Available: 103062476
Allocated Blocks: 866
SoundFonts: 1
Instruments: 14491
Samples: 864
Locked Instruments: 14491
Locked Samples: 864
A new sound font can be loaded with:
hansan@Desk-computer:~/work/sf2$ asfxload  "GeneralUser GS Live-Audigy v1.44.sf2"
hansan@Desk-computer:~/work/sf2$ asfxload -M
DRAM memory left = 100714 kB
And a midi file can be played with:
hansan@Desk-computer:~/work/sf2$ aplaymidi -l
 Port    Client name                      Port name
 14:0    Midi Through                     Midi Through Port-0
 16:0    PHASE 24 FW                      PHASE 24 FW MIDI 1
 28:0    SB Audigy 2 ZS [SB0350]          Audigy MPU-401 (UART)
 28:32   SB Audigy 2 ZS [SB0350]          Audigy MPU-401 #2
 29:0    Emu10k1 WaveTable                Emu10k1 Port 0
 29:1    Emu10k1 WaveTable                Emu10k1 Port 1
 29:2    Emu10k1 WaveTable                Emu10k1 Port 2
 29:3    Emu10k1 WaveTable                Emu10k1 Port 3
hansan@Desk-computer:~/work/sf2$ aplaymidi -p 29.0  test.mid
This does works at least.... But I have say that fluidsynth is maybe an even better solution, given the performance of modern processors.

Conclusion


All with all this was not really a good very invested 10.50 Euro. The selling features like the high bit rate and resolution are not really delivered on the linux platform.
The only good item is a firewire interface, which enables me to look into a different class of audio cards.


(updated and corrected a few items on 3/11/2018)

Sunday, 16 April 2017

Resurrection of a Lenovo Ideapad K1: to neon or not to neon

In my previous post here I discussed my struggle with flashing my wife’s Lenovo Ideapad K1. And in the end I had to conclude that I could not conclude that I have bad flash or bad memory. This all was interesting, but not so helpful to get the tablet back to usable again.

When I install my images:
  • First: CM10.1_Wifi_Kaschemme_v1 or K1_CM10_JB_Starter_v2 with the nvflash tool.
  • Second: cm-10.1.3-RELEASE-k1-wifi.zip or cm-10.1-20130916-NIGHTLY-k1.zip with the recovery firmware CWM6.0.x.x.
I don't have any problems. The problems start after installing the gapps that are missing in these CyanogenMod android 4.1 / 4.2 images.  I use for that the file gapps-jb-20130427-nonneon-tonyp-keyboard.zip

I did a bit more research on my problems. There are a few elements to my "not working" tablet which helped me to narrow down the issue:
  • Messages that various google services have stopped or are not working any more.
  • Programs like youtube that not start or google play that gives [RH01] errors.
  • Logfile that gets filled with SIGILL errors and register dumps.
The last symptom  helped me most. After some time spend with google search I started to understand that I am hitting here an essential Tegra 2 vs Android 3.2+ issue.

The Tegra 2 T20 was at the time a relative advanced dual core Cortex A9 processor with a very good GPU so that it could do video decoding and encoding in the GPU. The Arm part is good but the GPU was the more advanced Nvidia item. A bit similar situation like the BCM2835 SoC processor of the Raspberry pi. Only is this not an ARMv6 but real ARMv7-A architecture, but to reduce some costs/complexity they have not implemented all the typical Cortex-A9 functions like the Advanced SIMD (NEON) extension. Probably Nvidia expected not to need this because the GPU could do the multimedia power lifting much better. From Tegra 3 Nvidia starts to support the full "instruction set".

The SIGILL error happens when a program is trying to execute a NEON instruction on a not NEON processor.  In the past, older than Android 4.1, was the assumption that most processors were ARMv5 and did programs not use NEON instructions, also not for AMRv7A.  In the SDK was it also explicitly asked for to have libraries for ARMv5 and ARMv7A with and without NEON or other extensions.

The issue I now have in 2017 looks to be two fold:
  • Google started to ship APK packages optimized for NEON, maybe on purpose, maybe by accident or maybe the compiler starts to optimize more/better in this direction.
  • For higher android versions is NEON compatibility of the ARMv7A architecture assumed.
This means that when a newer Android is installed on a tegra 2 platform we will hit the lack of NEON support. The program or parts of it will crash when there are no backup libraries for NEON emulation. All the google service packages and programs such as the google play, youtube etc. have this (potential) problem. My issue is that this "newer" is already by the Android 4.2 from the CM10 images and that the gapps package is pulling in a lot of newer versions of the libraries to support certain API's to talk back to Google and the Android universe.

It must be possible to keep and skip to the correct versions of the google apps to keep them working. However that is not so easy. There is always the danger that a new installed app is bringing the system down.

At this moment I see as only option to stay at android 3.2 and not to update too much. I have installed the old not rooted official Lenovo image: K1_A301_11_11_111108_US.zip and with that older outdated version is for the last hours youtube and the web browser working.

Case closed: a working tablet with obsolete Android 3.2 and an odd ball ARM device in my collection.

A side note: this tablet went obsolete in early 2012 and therefore there are still some people working on it up to late 2013. There are quite some messages in the XDA and Lenove forums from that time, but nowadays are all links in the posts to file sharing servers dead. It is very difficult to find various images back. Therefore it is not really possible to have a long life on this kind devices in terms of (software) support. The PC is in that way a much durable concept.

Short update: Improved Android 3.2.1 image

I searched a bit longer and found here the following thread with a better Android 3.2.1. image for this tablet. This image has the following improvements:
  • Newer version of the root file system: 14_14_120109 build
  • Recovery image / bootloader
  • No levono market and other not working apps
  • Rooted
The original image name is: K1_Stock_120109_nvflash_R2.zip but it can also be found as honeycomb.zip

Saturday, 15 April 2017

Resurrection of a Lenovo Ideapad K1

My wife is the lucky owner of an Lenovo Ideapad K1, an Android 10" tablet I have bought for her in 2011.  Lenovo is clearly not supporting the product any more; an  difficult findable product on their own website.  This was a relative cool product, tegra 2 based and 1GB memory and 32 GB flash, that I could buy at the time in Japan. This tabled was a nice toy to consume media like reading websites of watching youtube movies.
However a few years back I got all kind of strange problems; programs crashed, the tablet didn't boot / come out of sleep  mode, etc.  Flashing a newer CyanogenMod firmware helped only for some time.
Recently I have taken the dust of the tablet to have a deeper look into the problems I am experiencing with the thing.

Lenovo IdeaPad K1 with crashes and errors in log file (CatLog)


Most information can be found in the XDA forums; this posting is a good start to find the required files.
I mainly used:
  • Recovery / first file system: nvflash_CM10.1_Wifi_Kaschemme_v1.zip
  • Main CyanogenMod firmware file system: cm-10.1-20130916-NIGHTLY-k1.zip and/or cm-10.1.3-RELEASE-k1-wifi.zip
  • Google apps items: gapps-jb-20130427-nonneon-tonyp-keyboard.zip

Detailed information

The following (among more) is inside:
  • An Nvidia Tegra 2 T20 processor at 1GHz.
  • The flash chip is Toshiba 32 GbB emmc chip with a now way obsolete 2nd generation controller in a painful to (de)solder BGA package: 32GB Toshiba THGBM2G8D8FBA1B NAND Flash.
  • The memory chip is also a BGA package: 1GB Eplida B8132B2PB-6D-F LPDDR2 DRAM.
Changing things in the hardware will be very hard and/or impossible.

The device has the following partition table:
Partition number Name Start sector Num Sectors Size Mbyte Description FS type Android / Linux mount point
2 BCT 0 1024 4 System configuration data data
3 PT 1024 512 2 Partition table data
4 EBT 1536 1024 4 Bootloader ? data
5 GP1 2560 512 2 also some bootloader related partition table data
6 SOS 3072 1536 6 Recovery partition android boot /dev/block/mmcblk0p1
7 LNX 4608 2048 8 kernel /boot partition android boot /dev/block/mmcblk0p2
8 APP 6656 107520 420 System root partition ext4 /dev/block/mmcblk0p3
/system
9 CAC 114176 32768 128 Cache partition ext4 /dev/block/mmcblk0p4
/cache
10 MSC 146944 512 2 miscellaneous data /dev/block/mmcblk0p5
11 USP 147456 35840 140 Bootloader ? data /dev/block/mmcblk0p6
12 UDA 183296 6740992 26332 User data ext3 /dev/block/mmcblk0p7
/data
13 TMP 6924288 614400 2400 Temp partition ext4 /dev/block/mmcblk0p8
14 GPT 7538688 276480 1080 Partition table data

The hope is that I can identify

Contacting the boot loader

One of the important steps is to get "contact" with the bootloader to get a deeper understanding of the state of the flash.
The tegra has a possibility to boot over USB. That means that the ROM bootloader is not getting its further data from flash (nand, sd, emmc), but waits on data coming from the USB subsystem. This is a special protocol of Nvidia.

The linux tool for this is called nvflash, this program can be found in the archive of the upgrade images. More information can be found here or with google.

The first step in the process is to get a more "advanced" bootloader working on the process so that actually communication and flashing is possible. For this is a special "Nv3p server" bootloader needed (bootloader.bin).  Because of the secure system should this bootloader  be accompanied by a key in a binary blob (blob.bin).  And there is the need for a kind of board description file BCT file (flash.bct) to let the bootloader know what kind of memory and flash is used. This all can be found in the flash package for android.


Making an udev rule

By making an udev rule is it possible to set the correct permissions and to remove the need to be root to communicate with the tablet. This is actually linux basics, but not something I do daily so I personally forget all the details every time.

Basic work flow:
Check location of the device with lsusb:
$ lsusb
Bus 003 Device 012: ID 0955:7820 NVidia Corp. Tegra 2 AC100 developer mode

Check the relevant parameters for the udev script:
$ udevadm info -a -p $(udevadm info -q path -n /dev/bus/usb/003/012)
  looking at device '/devices/pci0000:00/0000:00:14.0/usb3/3-3/3-3.1':
    KERNEL=="3-3.1"
    SUBSYSTEM=="usb"
    DRIVER=="usb"

    ATTR{idProduct}=="7820"
    ATTR{idVendor}=="0955"
    ATTR{ltm_capable}=="no"
    ATTR{manufacturer}=="NVIDIA Corp."
    ATTR{maxchild}=="0"

Make the udev rule:

$cat /etc/udev/rules.d/60-nvflash.rules 
# nvflash support for Nvidia Tegra systems attached to USB
SUBSYSTEM=="usb", ATTR{idVendor}=="0955", ATTR{idProduct}=="7820", MODE="0660", GROUP="plugdev"

Reload the rules and check if something is recognized:

$ sudo udevadm control --reload-rules 
$ udevadm test /devices/pci0000:00/0000:00:14.0/usb3/3-3/3-3.1
$ sudo udevadm monitor 

Getting data from the flash chip


With the following commands is it possible to get the partition table and the data of each partition. The first command ensures that the Nv3p server is up and running.

$ ./nvflash --blob blob.bin --bct flash.bct --odmdata 0x300c0011 --bl bootloader.bin --getpartitiontable partition.txt
$ ./nvflash --resume read 12 UDA.bin
$ ./nvflash --resume read 3 PT.bin

This did not result into an error. So reading from flash does work in principle. Next is a sanity check of each partition, where the data partitions are most important.

$ file UDA.bin
UDA.bin: Linux rev 1.0 ext3 filesystem data, UUID=1d1ae925-25c3-4daf-a78c-5e92106cad73 (large files)
$ fsck.ext3 -v -f UDA.bin
$ mkdir -p test
$ sudo mount UDA.bin test

Here are errors found. I am not certain if this means we have errors on our flash chip or that the file system just crashed by a hard shutdown before.

Re-flashing the device

I try to flash the image with the verify option switched on.

$ ./nvflash --blob blob.bin --bct flash.bct --odmdata 0x300c0011 --configfile flash.cfg --bl bootloader.bin --format_partition 2 --sync
$ ./nvflash -r --format_partition 2
$ ./nvflash -r --format_partition 3
          ....
$ ./nvflash -r --format_partition 14
$ ./nvflash -r --blob blob.bin --bct flash.bct --setbct --odmdata 0x300c0011 --configfile flash.cfg --create --verifypart -1
Nvflash v1.8.90246 started
[resume mode]
Using blob v1.1.57813
sending file: flash.bct
- 4080/4080 bytes sent
flash.bct sent successfully
odm data: 0x300c0011
setting device: 2 3
creating partition: BCT
creating partition: PT
creating partition: EBT
creating partition: GP1
creating partition: SOS
creating partition: LNX
creating partition: APP
creating partition: CAC
creating partition: MSC
creating partition: USP
creating partition: UDA
creating partition: TMP
creating partition: GPT
Formatting partition 2 BCT please wait.. done!
Formatting partition 3 PT please wait.. done!
Formatting partition 4 EBT please wait.. done!
Formatting partition 5 GP1 please wait.. done!
Formatting partition 6 SOS please wait.. done!
Formatting partition 7 LNX please wait.. done!
Formatting partition 8 APP please wait.. done!
Formatting partition 9 CAC please wait.. done!
Formatting partition 10 MSC please wait.. done!
Formatting partition 11 USP please wait.. done!
Formatting partition 12 UDA please wait.. done!
Formatting partition 13 TMP please wait.. done!
Formatting partition 14 GPT please wait.. done!
done!
Enabling verification for partition ID = 4...
sending file: bootloader.bin
\ 1393296/1393296 bytes sent
bootloader.bin sent successfully
Enabling verification for partition ID = 6...
sending file: recovery.img
\ 4573184/4573184 bytes sent
recovery.img sent successfully
Enabling verification for partition ID = 7...
sending file: boot.img
| 2816000/2816000 bytes sent
boot.img sent successfully
Enabling verification for partition ID = 8...
sending file: system.img
- 264010336/264010336 bytes sent
system.img sent successfully
Enabling verification for partition ID = 13...
sending file: userdata.img
And for some reason is the process hanging at this point. The userdata.img file is smaller than the system.img file; I would expect that the flash process goes with the same speed or faster. Is this than the sign that flash or memory is bad? The tablet feels warm so maybe it is from calculating the checksum; I will have to wait for an hour.
Waiting is not really working. When the verification option is enabled is the userdata.img always failing around 139657210 bytes. This does not become better when I move partition 13 a significant amount of sectors forward. Probably this means I have a memory problem and not a flash problem.
A normal write, without the checksum calculation for the verify, does work normally.

$ ./nvflash --bl bootloader.bin --blob blob.bin --bct flash.bct --setbct --odmdata 0x300c0011 --configfile flash.cfg --create 
Nvflash v1.8.90246 started
Using blob v1.1.57813
rcm version 0X20001
System Information:
   chip name: t20
   chip id: 0x20 major: 1 minor: 4
   chip sku: 0x8
   chip uid: 0x0428914840c12157
   macrovision: disabled
   hdcp: enabled
   sbk burned: true
   dk burned: true
   boot device: emmc
   operating mode: 4
   device config strap: 0
   device config fuse: 0
   sdram config strap: 0

sending file: flash.bct
- 4080/4080 bytes sent
flash.bct sent successfully
odm data: 0x300c0011
downloading bootloader -- load address: 0x108000 entry point: 0x108000
sending file: bootloader.bin
\ 1393296/1393296 bytes sent
bootloader.bin sent successfully
waiting for bootloader to initialize
bootloader downloaded successfully
setting device: 2 3
creating partition: BCT
creating partition: PT
creating partition: EBT
creating partition: GP1
creating partition: SOS
creating partition: LNX
creating partition: APP
creating partition: CAC
creating partition: MSC
creating partition: USP
creating partition: UDA
creating partition: TMP
creating partition: GPT
Formatting partition 2 BCT please wait.. done!
Formatting partition 3 PT please wait.. done!
Formatting partition 4 EBT please wait.. done!
Formatting partition 5 GP1 please wait.. done!
Formatting partition 6 SOS please wait.. done!
Formatting partition 7 LNX please wait.. done!
Formatting partition 8 APP please wait.. done!
Formatting partition 9 CAC please wait.. done!
Formatting partition 10 MSC please wait.. done!
Formatting partition 11 USP please wait.. done!
Formatting partition 12 UDA please wait.. done!
Formatting partition 13 TMP please wait.. done!
Formatting partition 14 GPT please wait.. done!
done!
sending file: bootloader.bin
\ 1393296/1393296 bytes sent
bootloader.bin sent successfully
sending file: recovery.img
\ 4573184/4573184 bytes sent
recovery.img sent successfully
sending file: boot.img
| 2816000/2816000 bytes sent
boot.img sent successfully
sending file: system.img
- 264010336/264010336 bytes sent
system.img sent successfully
sending file: userdata.img
- 142382496/142382496 bytes sent
userdata.img sent successfully
This probably means that I will have to search for a memtest tool for the tegra.
The tool to check the sdram didn't work for me. Nvidia has some special tools but they are not free available or ask for a special memory description files.  I can maybe get those out of the bct file but still...
The output below is what the nvflash tool is giving me:

$ ./nvflash --cmdhelp --odm
Nvflash v1.8.90246 started
--odm command usage ->
nvflash --odm CMD data --bl  --go
-----------------------------------------------------------------------
used to do some special diagnostics like sdram validation, fuelgauge etc
each CMD has data associated with it
For e.g.
1. SD/SE diagnostic tests:
   nvflash --odm runsediag  --bl  --go
   nvflash --odm runsddiag  --bl  --go
   ---------------------------------------------------------------------
   used to run SD/SE diagnostic tests, where N is the input parameter like
   SDController Instance.
2. Sdram verification:
   nvflash --bct  --setbct --configile  --create --odmdata     --odm verifysdram  --bl  --go
   ---------------------------------------------------------------------
   used to verify sdram initialization i.e. verify sdram params in bct is
   in accordance with actual SDRAM attached in device or not.
   N can be either 0 or 1. '0' implies soft test where known word patterns
   are written for each mb and read back. This word pattern validates all
   the bits of the word. '1' implies stress test where known word patterns
   are written on entire sdram area available and read back. This tests every
   bit of the the entire available sdram area
-----------------------------------------------------------------------
$ ./nvflash -r --bct flash.bct --configfile flash.cfg --odmdata 0x300c0011 --odm verifysdram 0 
Nvflash v1.8.90246 started
[resume mode]
command failure: no commands found

Conclusion

I cannot really conclude that my memory or flash is bad, despite of the "struggle" I had with the nvflash tool. In general I don't see errors with the flash process, and when I have an error then they don't depend on the flash location and are probably more a not related "feature" of the nvflash tool and bootloader firmware.
A memory error would be still possible, but in that case I would have expected more often problems and real hard crashes.
The conclusion must be that I have a different problem.

Tuesday, 22 November 2016

Updating DVD drive firmware

I recently  tried to watch a few DVD's with the drive in my computer. That didn't work so well; towards the end of the "file" was the drive not able to read the frames any more. Maybe this has something to do with the second layer on the disk or something with the security on a DVD.
But anyhow I thought it was maybe a good thing to try to update the firmware  of the DVD drive to see if that helps.

I have, as Linux advertises, a HL-DT-ST DVDRAM GSA-H12N in my linux computer. This is relative old LG made drive with a Pata interface that can read and write DVDs next to CDs.

 Looking for a firmware upgrade showed only a windows utilities for firmware upgrades. The new firmware was packed in "GSAH12N_UL02.exe".  I couldn't get it working under Freedos, because it explicitly needed windows. And with wine I only got an error message / no response at all. But I had not much confidence in wine for this hardware job, which made that I started to do some Google search.
 
This question on launchpad gave me a hint how I could get it working. I did needed to install the mfc42.dll with help of winetricks.

Then I only needed to do the following to update the firmware:

sudo chown -R root ~/.wine
sudo wine GSAH12N_UL02.exe
sudo chown -R $USER:$USER ~/.wine


I did had to add a disk to the drive to make sure that the windows program could find it.

After the reboot is my DVD drive indeed advertising the higher firmware number UL01 is now UL02. However I am still not able to read the complete DVD. That will be the next thing to sort-out.



Sunday, 2 October 2016

Finding a use for an obsolete tablet: part 1 - resurrection

Somewhere in the year 2012 I thought buying a cheapish Chinese android based tablet was a good idea. My wife just got her first iPad, and I was curious what sort of tablet €85 would get me. From a tablet point of view this turned out as somewhat of a disappointment, or rather, it would have been a disappointment if I would have expected to get a useable tablet.

Looking at it from the hackable Linux device perspective it was actually quite okay. It came with a 7" 800x480 colour LCD, ethernet, WiFi, serial port, microphone, speakers, camera, two USB ports, 2Gb flash, a micro SD port, and a resistive (i.e. single touch) touchscreen digitiser. On top of that it had a rechargeable battery that would sustain the thing for about two hours. And all this rather underpowered by a 300MHz ARM processor and 128Mb of ram.
The tablet came with this dongle. On this side I hacked a serial port. On the other side, there are two USB2 ports. And on the left side, there is an UTP plug for ethernet.
Unfortunately, I started with trying different versions of Android based firmware. A firmware upgrade consisted of downloading a zip, extracting to SD card, and booting the tablet with the SD in it. Apart from a new android kernel and rootfs, all these firmware updates always included an update for w-boot and u-boot as well. These were the pre-bootloader and main bootloader respectively. At a certain moment I left the SD card in when I hadn't meant to, booted, noticed my mistake, yanked the card out, and corrupted either w-boot or u-boot. As a result, the tablet would not boot anymore, and I had made myself a fancy paperweight.

At the time, I looked into fixing this. The bootloaders are stored in a spi flash chip, while the rest of the software lives in another set of flash chips. So I hooked my buspirate to the spi flash and tried to access it in circuit. This was ultimately unsuccessful: while I could read the flash chip id, the rest of the tablet's circuitry a somehow prevented me from writing or even reliably reading the chip's contents. I decided that I would have to desolder the chip to continue, and that I had more important things to. 
Attempt at in-situ programming of the spi flash chip.

So I put the tablet in its box and more or less forgot about it for a couple of years.

Recently I came across it again while looking for something else. I then saw connection points for a JTAG interface. These I had noticed before, but at the time I did not have anything to interface with JTAG. But now it seemed that, with a firmware update, my buspirate and openocd could be made to do this.

So I soldered a connector to the JTAG interface, hooked it to my buspirate and tried openocd. I quickly found that openocd needs some kind of description of the item on the other side of the JTAG to work. This makes sense as designing such a hardware debug interface, with a very limited number of users, for  "plug and play" would be a serious waste of effort. Unfortunately the tablet was based on a wondermedia WM8505-board that was unknown to openocd.

JTAG header added to tablet. GND is hooked to the spi flash ground pin. And I stuck a description of the pins to the inside of the tablet case, for future reference.
This meant I had to make my own definition file, while discovering how openocd actually works. I could find a single post on a forum of someone who had done the same or similar, but that post did not have much detail. But fortunately, it all proved relatively straightforward. When I connected the board, only a single JTAG tap (something like a client or receiver) was found, so this had to be the processor. And by now I knew that the WM8505 had an ARM926ej-s core, for which openocd already had a definition file. So after some experimenting, I could halt the processor on the tablet, dump/change its registers and other state, and make it do anything I want.

For those in need of this, here is the definition file I cobbled together:
# Barebones openocd definition file for WM8505
#
# not sure if this actually works
jtag_rclk 3000

# WM8505 SoC
jtag newtap WM cpu -irlen 4 -expected-id 0x07926f0f

# CPU
target create CPU arm926ejs -chain-position WM.cpu

# no hard reset, use soft reset
CPU configure -event reset-assert  { jtag arp_init }

# Work area in ram: last 8k from 128M
CPU configure -work-area-phys 0x07ffe000 -work-area-size 0x4000

arm7_9 dcc_downloads enable
arm7_9 fast_memory_access enable

reset_config trst_only
Next step was to load a copy of u-boot via JTAG into the tablet's ram, and run it. From there, I could take control of u-boot via the serial interface, and use that to flash both u-boot and w-boot from the SD card into the spi rom. But first I hit a small snag: the first 6 or so u-boot versions I tried all seemed to work, but then shut down the tablet. Fortunately, the 7th version did give me a prompt, and I could fix the corrupted spi flash. 

(I later discovered that the other u-boot copies were actually fine: what happened was that these checked the state of the power-button. If the button was pressed for about 1 second, the boot process would continue. If not, the tablet would power down. As u-boot is (almost) the first thing to run when the tablet is powered on, this would prevent accidental turning on of the tablet. But in my JTAG case, the power button was always off, so u-boot would power down the tablet.)

After fixing the spi flash, it was also possible again to update the original android firmware in nand flash. However, I had difficulties finding a correct one. For starters I  was and am not 100% sure which exact model of tablet I have. Many variants with minor differences in hardware existed, and firmwares of other models would sort-of work on mine, but with some features (i.e. touchscreen or audio) missing. And most firmware versions originally were on download sites that either no longer exist, or do not have the files anymore. Some of the versions that I could find did boot and show something but I could not get the touchscreen to work.

Success! Notice the low-glare screen on this high quality product.
So I decided to forget about android, and install a plain Debian Linux on the tablet. But that will be the subject of another post.