Styjun

Edit: a follow-up question: Restore mongoDB by --repair and WiredTiger.

My developer committed a huge mistake and we cannot find our Mongo database anywhere in the server.

He logged into the server, and saved the following shell under ~/crontab/mongod_back.sh:

#!/bin/sh
DUMP=mongodump
OUT_DIR=/data/backup/mongod/tmp // 备份文件临时目录
TAR_DIR=/data/backup/mongod // 备份文件正式目录
DATE=`date +%Y_%m_%d_%H_%M_%S` // 备份文件将以备份对间保存
DB_USER=Guitang // 数库操作员
DB_PASS=qq■■■■■■■■■■■■■■■■■■■■■ // 数掘库操作员密码
DAYS=14 // 保留最新14天的份
TARBAK="mongod_bak_$DATE.tar.gz" // 备份文件命名格式
cd $OUT_DIR // 创建文件夹
rm -rf $OUT_DIR/* // 清空临时目录
mkdir -p $OUT_DIR/$DATE // 创建本次备份文件夹
$DUMP -d wecard -u $DB_USER -p $DB_PASS -o $OUT_DIR/$DATE // 执行备份命令
tar -zcvf $TAR_DIR/$TAR_BAK $OUT_DIR/$DATE // 将份文件打包放入正式
find $TAR_DIR/ -mtime +$DAYS -delete // 除14天前的旧备

And then he ran it and it outputted permission denied messages, so he pressed Ctrl+C. The server shut down automatically. He tried to restart it but got a grub error:

He contacted AliCloud, the engineer connected the disk to another working server so that he could check the disk. Looks like some folders are gone, including /data/ where the mongodb is!

1. We don't understand how the script could destroy the disk including /data/;


2. And of course, is it possible to get the /data/ back?

PS: He did not take snapshot of the disk before.

PS2: As people mention "backups" a lot, we have lots of important users and data coming these 2 days, the purpose of this action was to backup them (for the first time), then they turned out to be entirely deleted.

Easy enough. The // sequence isn't a comment in bash (# is).

The statement OUT_DIR=x // text had no effect* except a cryptic error message.

Thus, with the OUT_DIR being an empty string, one of the commands eventually executed was rm -rf /*. Some directories placed directly underneath / weren't removed due to user not having permissions, but it appears that some vital directories were removed. You need to restore from backup.

* The peculiar form of bash statement A=b c d e f is roughly similar to:

export A=b
c d e f
unset A

A common example:

export VISUAL=vi # A standard visual editor to use is `vi`
visudo -f dummy_sudoers1 # Starts vi to edit a fake sudo config. Type :q! to exit
VISUAL=nano visudo -f dummy_sudoers2 # Starts nano to edit a fake sudo config
visudo -f dummy_sudoers3 # Starts vi again (!)

And the problematic line of script amounted to this:

export OUT_DIR=/data/backup/mongod/tmp
// 备份文件临时目录 # shell error as `//` isn't an executable file!
unset OUT_DIR

1) He erroneously assumed that // was a bash comment. It is not, only # is.

The shell interpreted // text as a normal command, and did not find a binary called //, and did nothing.

In bash, when you have a variable assignment (OUT_DIR=/data/backup/mongod/tmp) directly preceding a command (// text), it only sets the variable while running the command. Therefore, it unsets OUT_DIR immediately, and when the rm line is reached, OUT_DIR is now unset, and rm -rf / is now called, deleting everything you have permission to delete.

2) The solution is the same as all rm -rf / cases: restore from backup. There is no other solution because you do not have physical access to the hard drive.

1) Bash comments start with #. Sorry for your loss.
2) Restore from backup is the only way to proceed here, unfortunately.