为了账号安全,请及时绑定邮箱和手机立即绑定

ART虚拟机是如何内建支持MultiDex的?

标签:
Android

  作为一个合格的程序员,你一定不会错过这个异常:


1

2

3

4

5

6

7

8

9

10

11

NEXPECTED TOP-LEVEL EXCEPTION: java.lang.IllegalArgumentException: method ID not in [0, 0xffff]: 65536

    at com.android.dx.merge.DexMerger$6.updateIndex(DexMerger.java:501)

    at com.android.dx.merge.DexMerger$IdMerger.mergeSorted(DexMerger.java:276)

    at com.android.dx.merge.DexMerger.mergeMethodIds(DexMerger.java:490)

    at com.android.dx.merge.DexMerger.mergeDexes(DexMerger.java:167)

    at com.android.dx.merge.DexMerger.merge(DexMerger.java:188)

    at com.android.dx.command.dexer.Main.mergeLibraryDexBuffers(Main.java:439)

    at com.android.dx.command.dexer.Main.runMonoDex(Main.java:287)

    at com.android.dx.command.dexer.Main.run(Main.java:230)

    at com.android.dx.command.dexer.Main.main(Main.java:199)

    at com.android.dx.command.Main.main(Main.java:103):Derp:dexDerpDebug FAILED


项目的代码量越来越大,引入的jar越来越多,由于Dalvik虚拟机作者当初对Method量的短视,最终造成了Dex方法数超标的悲剧。Google为此提供了MultiDex这一补丁方案。

  MultiDex分为两部分,一部分是编译时需要的IDE插件,它负责将单个的classes.dex拆分成多个dex文件;而另一部分则是编译进classes.dex的运行时环境,它将classes2.dex, classes3.dex…在运行时加载进来,从而拼合成完整的字节码。

  作为一个补丁方案,它一定会被慢慢的取代,果不其然,随着ART虚拟机的诞生,MultiDex的内建支持出现了,接下来我们就来分析ART模式下是如何内建支持MultiDex的。

  想要弄清楚ART下内建支持MultiDex的机制,首先需要知道从哪个版本的虚拟机开始支持这一特征,我们从MultiDex源码里找到了答案:


1

2

3

4

5

6

7

    public static void install(Context context) {

        if (IS_VM_MULTIDEX_CAPABLE) {

            Log.i(TAG, "VM has multidex support, MultiDex support library is disabled.");

            return;

        }

        //...

}


在这里,当IS_VM_MULTIDEX_CAPABLE条件为真时,install函数直接返回了。从log的含义中我们可以知道,当虚拟机内建支持MultiDex,运行库将直接返回。它的赋值在这里:


1

2

  private static final boolean IS_VM_MULTIDEX_CAPABLE =

            isVMMultidexCapable(System.getProperty("java.vm.version"));


于是,一切的矛头指向了isVMMultidexCapable这个函数:


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

    /**

     * Identifies if the current VM has a native support for multidex, meaning there is no need for

     * additional installation by this library.

     * @return true if the VM handles multidex

     */

    /* package visible for test */

    static boolean isVMMultidexCapable(String versionString) {

        boolean isMultidexCapable = false;

        if (versionString != null) {

            Matcher matcher = Pattern.compile("(\\d+)\\.(\\d+)(\\.\\d+)?").matcher(versionString);

            if (matcher.matches()) {

                try {

                    int major = Integer.parseInt(matcher.group(1));

                    int minor = Integer.parseInt(matcher.group(2));

                    isMultidexCapable = (major > VM_WITH_MULTIDEX_VERSION_MAJOR)

                            || ((major == VM_WITH_MULTIDEX_VERSION_MAJOR)

                                    && (minor >= VM_WITH_MULTIDEX_VERSION_MINOR));

                } catch (NumberFormatException e) {

                    // let isMultidexCapable be false

                }

            }

        }

        Log.i(TAG, "VM with version " + versionString +

                (isMultidexCapable ?

                        " has multidex support" :

                        " does not have multidex support"));

        return isMultidexCapable;

    }


它做了两件事情:

1、通过正则表达式将版本号分成major(主版本号)和minor(次版本号)。

2、通过判断主版本和次版本是否大于一个常量来判定虚拟机内建支持MultiDex。

  对应的常量定义如下:


1

2

    private static final int VM_WITH_MULTIDEX_VERSION_MAJOR = 2;

    private static final int VM_WITH_MULTIDEX_VERSION_MINOR = 1;


由此得出,当虚拟机的主版本号大于3或版本大于等于2.1时,就意味着内建支持MultiDex。实际中,不同Android版本的虚拟机版本对照表如下:

              Android版本  |   虚拟机版本

               android 4.4  |     2.0

               android 5.0  |     2.1

            android 5.0.1  |     2.1

               android 5.1  |     2.1

               android 6.0  |     2.1

  由表可知,4.4以后的ART虚拟机均支持内建的MultiDex特征。可能有同学很好奇,为什么4.4的虚拟机无法支持这一特征呢?原因很简单,4.4的ART虚拟机还处于测试阶段,稳定都还没稳定下来,还怎么支持其它的高级特征呢?

理解了这些,我们就可以深入探索ART虚拟机内部的实现了。在Android的类加载器体系中,无论是PathClassLoader还是DexClassLoader,它们都支持传入一个APK,从APK的内部直接加载classes.dex, 而它们的dex字节码加载都是在dalvik.system.DexFile的JNI方法中实现的,因此DexFile就是根入口。


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

/**

     * Opens a DEX file from a given filename, using a specified file

     * to hold the optimized data.

     *

     * @param sourceName

     *  Jar or APK file with "classes.dex".

     * @param outputName

     *  File that will hold the optimized form of the DEX data.

     * @param flags

     *  Enable optional features.

     */

    private DexFile(String sourceName, String outputName, int flags) throws IOException {

        if (outputName != null) {

            try {

                String parent = new File(outputName).getParent();

                if (Libcore.os.getuid() != Libcore.os.stat(parent).st_uid) {

                    throw new IllegalArgumentException("Optimized data directory " + parent

                            + " is not owned by the current user. Shared storage cannot protect"

                            + " your application from code injection attacks.");

                }

            } catch (ErrnoException ignored) {

                // assume we'll fail with a more contextual error later

            }

        }

        mCookie = openDexFile(sourceName, outputName, flags);

        mFileName = sourceName;

        guard.open("close");

    }


DexFile的构造器中三个参数含义如下:

sourceName   :   Apk/Dex路径

outputName   :   dex优化后的输出路径

flags               :   决定dex缓存如何处理的标志

这里还做了一件很有意思的事情,通过Libcore.os来对dex输出路径的所属uid进行检查,当目录的所属uid不  属于当前app时立刻抛出异常。这段代码在4.0以后就被加入进来了,这么做的目的是防止优化后的dex文件被存放到不安全的地方,从而病毒对字节码进行注入。

  一切无误后,接着调用openDexFile函数,下面是openDexFile的实现:


1

2

3

4

5

6

7

8

9

10

11

12

    /*

     * Open a DEX file.  The value returned is a magic VM cookie.  On

     * failure, an IOException is thrown.

     */

    private static int openDexFile(String sourceName, String outputName,

        int flags) throws IOException {

        return openDexFileNative(new File(sourceName).getCanonicalPath(),

                                 (outputName == null) ? null : new File(outputName).getCanonicalPath(),

                                 flags);

    }

    native private static int openDexFileNative(String sourceName, String outputName,

        int flags) throws IOException;


  如注释所说,openDexFile返回的是int类型的magic VM cookie,其实质是一个native指针,当返回值为0的时候,就表示返回了一个空指针,即native层没有成功打开Dex文件。

  那么native层又是怎样实现的呢?我们找到 art/runtime/native/dalvik_system_DexFile.cc。


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

jstring javaOutputName, jint) {

  ScopedUtfChars sourceName(env, javaSourceName);//Dex或APK路径

  if (sourceName.c_str() == NULL) {

    return 0;

  }

  NullableScopedUtfChars outputName(env, javaOutputName);//优化后的Dex路径

  if (env->ExceptionCheck()) { //侦测到有异常抛出

    return 0;

  }

  ClassLinker* linker = Runtime::Current()->GetClassLinker();//取得类链接器

  

  std::unique_ptr<std::vector<const DexFile*>> dex_files(new std::vector<const DexFile*>());//Dex文件列表

  std::vector<std::string> error_msgs;//每一个Dex加载后的异常信息,没有异常为空

  bool success = linker->OpenDexFilesFromOat(sourceName.c_str(), outputName.c_str(), &error_msgs,

                                             dex_files.get());

  if (success || !dex_files->empty()) {

    //...

  } else {

    //...

  }

}


  代码十分清晰,首先把传入的Dex/Apk路径和优化路径转换为native字符串,预先创建好存放DexFile的Vector,然后通过 ClassLinker::openDexFileFromOat 来打开Dex/Apk。

  代码读到了这里,读者有没有看出一点门道,没有的话请认真思考一下,为什么 ClassLinker::openDexFileFromOat 传入的第四个参数是一个Vector指针,而不是单一的DexFile指针呢?这显然说明虚拟机能够正确的认识到APK中存在有多个dex文件。看来我们快要接近BOSS了,怀着激动的心情,我们来到 art/runtime/class_linker.cc。


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

// Multidex files make it possible that some, but not all, dex files can be broken/outdated. This

// complicates the loading process, as we should not use an iterative loading process, because that

// would register the oat file and dex files that come before the broken one. Instead, check all

// multidex ahead of time.

bool ClassLinker::OpenDexFilesFromOat(const char* dex_location, const char* oat_location,

                                      std::vector<std::string>* error_msgs,

                                      std::vector<const DexFile*>* dex_files) {

 

  uint32_t dex_location_checksum;

  uint32_t* dex_location_checksum_pointer = &dex_location_checksum;

  bool have_checksum = true;

  std::string checksum_error_msg;

  //检查Sum值

  if (!DexFile::GetChecksum(dex_location, dex_location_checksum_pointer, &checksum_error_msg)) {

    dex_location_checksum_pointer = nullptr;

    have_checksum = false;

  }

//查看目标Oat文件是否已经被加载过了

  const OatFile::OatDexFile* oat_dex_file = FindOpenedOatDexFile(oat_location, dex_location,

                                                                 dex_location_checksum_pointer);

  std::unique_ptr<const OatFile> open_oat_file(

      oat_dex_file != nullptr ? oat_dex_file->GetOatFile() : nullptr);

  if (open_oat_file.get() == nullptr) {

    //异常处理

    //...

  }

  bool success = LoadMultiDexFilesFromOatFile(open_oat_file.get(), dex_location,

                                              dex_location_checksum_pointer,

                                              false, error_msgs, dex_files);

  if (success) {

    //...

  } else {

    //...

  }

//...

}


  看了函数的实现,可不要被它的函数名迷惑了,这里所谓的 oat_location 是指该APK优化后的OAT文件的路径,OAT文件的实质是一个ELF文件,但在其内部可以同时保存有多个dex文件,即一个oat文件可以对应多个dex文件,这也恰恰符合MultiDex的思想,我们接着看ClassLinker::LoadMultiDexFilesFromOatFile


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

// Loads all multi dex files from the given oat file returning true on success.

//

// Parameters:

//   oat_file - the oat file to load from

//   dex_location - the dex location used to generate the oat file

//   dex_location_checksum - the checksum of the dex_location (may be null for pre-opted files)

//   generated - whether or not the oat_file existed before or was just (re)generated

//   error_msgs - any error messages will be appended here

//   dex_files - the loaded dex_files will be appended here (only if the loading succeeds)

static bool LoadMultiDexFilesFromOatFile(const OatFile* oat_file,

                                         const char* dex_location,

                                         const uint32_t* dex_location_checksum,

                                         bool generated,

                                         std::vector<std::string>* error_msgs,

                                         std::vector<const DexFile*>* dex_files) {

  if (oat_file == nullptr) {

    return false;

  }

  size_t old_size = dex_files->size();

  bool success = true;

  //这个for循环执行的过程中,一旦出现success为false,立刻停止循环

  for (size_t i = 0; success; ++i) {

    std::string next_name_str = DexFile::GetMultiDexClassesDexName(i, dex_location);

    const char* next_name = next_name_str.c_str();

    

    std::string error_msg;

    //...

    //尝试寻找对应的OAT文件

    const OatFile::OatDexFile* oat_dex_file = oat_file->GetOatDexFile(next_name, nullptr, false);

    if (oat_dex_file == nullptr) {

      if (i == 0 && generated) {

        std::string error_msg;

        error_msg = StringPrintf("\nFailed to find dex file '%s' (checksum 0x%x) in generated out "

                                 " file'%s'", dex_location, next_location_checksum,

                                 oat_file->GetLocation().c_str());

        error_msgs->push_back(error_msg);

      }

      break;  // Not found, done.

    }

    // Checksum test. Test must succeed when generated.

    success = !generated;

    //...

    if (success) {

      const DexFile* dex_file = oat_dex_file->OpenDexFile(&error_msg);

      if (dex_file == nullptr) {

        success = false;

        error_msgs->push_back(error_msg);

      } else {

        dex_files->push_back(dex_file);

      }

    }

    

  }//for循环结束

  if (dex_files->size() == old_size) {

    success = false;  // We did not even find classes.dex

  }

  if (success) {

    return true;

  } else {

    // Free all the dex files we have loaded.

    auto it = dex_files->begin() + old_size;

    auto it_end = dex_files->end();

    for (; it != it_end; it++) {

      delete *it;

    }

    dex_files->erase(dex_files->begin() + old_size, it_end);

    return false;

  }

}


从函数名我们就看到,MultiDex出现了,在这个函数中有一个for循环,但是很遗憾,它跟MultiDex无关,当加载的是一个APK时,这个for循环只循环了一次,ClassLinker::GetMultiDexClassesDexName 在这里返回的是classes.dex,并没有对多个dex加载的实现。那么到底哪里才会是加载classes2.dex、classes3.dex…的地方呢? 我们把目光对向了 oat_dex_file->OpenDexFile(&error_msg) 这句话。OatDexFile::OpenDexFile 位于 art/runtime/oat_file.cc。


1

2

3

4

const DexFile* OatFile::OatDexFile::OpenDexFile(std::string* error_msg) const {

  return DexFile::Open(dex_file_pointer_, FileSize(), dex_file_location_,

                       dex_file_location_checksum_, error_msg);

}


  它直接调用了DexFile::Open,我们继续跟踪,找到 art/runtime/dex_file.cc :


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

bool DexFile::Open(const char* filename, const char* location, std::string* error_msg,

                   std::vector<const DexFile*>* dex_files) {

  uint32_t magic;

  ScopedFd fd(OpenAndReadMagic(filename, &magic, error_msg));

  if (fd.get() == -1) {

    DCHECK(!error_msg->empty());

    return false;

  }

  if (IsZipMagic(magic)) {

   //因为传入的是APK,所以进到这里

    return DexFile::OpenZip(fd.release(), location, error_msg, dex_files);

  }

  if (IsDexMagic(magic)) {

    std::unique_ptr<const DexFile> dex_file(DexFile::OpenFile(fd.release(), location, true,

                                                              error_msg));

    if (dex_file.get() != nullptr) {

      dex_files->push_back(dex_file.release());

      return true;

    } else {

      return false;

    }

  }

  *error_msg = StringPrintf("Expected valid zip or dex file: '%s'", filename);

  return false;

}


  在这里,会通过读取文件的magic值判断文件的类型,假如文件是一个APK的话,就以压缩包的方式打开文件,不然则以Dex文件方式打文件,由于这里的filename依旧是最初的APK路径,故假如一切无误的话,流程走到 DexFile::OpenZip :


1

2

3

4

5

6

7

8

9

bool DexFile::OpenZip(int fd, const std::string& location, std::string* error_msg,

                      std::vector<const  DexFile*>* dex_files) {

  std::unique_ptr<ZipArchive> zip_archive(ZipArchive::OpenFromFd(fd, location.c_str(), error_msg));

  if (zip_archive.get() == nullptr) {

    DCHECK(!error_msg->empty());

    return false;

  }

  return DexFile::OpenFromZip(*zip_archive, location, error_msg, dex_files);

}


  这个函数尝试从fd(文件描述符)中打开ZipArchive,然后调用DexFile::OpenFromZip。是不是有一种快要看到庐山真面目的感觉?我们跟踪到OpenFromZip :


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

bool DexFile::OpenFromZip(const ZipArchive& zip_archive, const std::string& location,

                          std::string* error_msg, std::vector<const DexFile*>* dex_files) {

  ZipOpenErrorCode error_code;

  std::unique_ptr<const DexFile> dex_file(Open(zip_archive, kClassesDex, location, error_msg,

                                               &error_code));

  if (dex_file.get() == nullptr) {

    return false;

  } else {

    // Had at least classes.dex.

    dex_files->push_back(dex_file.release());

    // Now try some more.

    size_t i = 2;

    // We could try to avoid std::string allocations by working on a char array directly. As we

    // do not expect a lot of iterations, this seems too involved and brittle.

    while (i < 100) {

      std::string name = StringPrintf("classes%zu.dex", i);

      std::string fake_location = location + kMultiDexSeparator + name;

      std::unique_ptr<const DexFile> next_dex_file(Open(zip_archive, name.c_str(), fake_location,

                                                        error_msg, &error_code));

      if (next_dex_file.get() == nullptr) {

        if (error_code != ZipOpenErrorCode::kEntryNotFound) {

          LOG(WARNING) << error_msg;

        }

        break;

      } else {

        dex_files->push_back(next_dex_file.release());

      }

      i++;

    }

    return true;

  }

}


  来到这个函数,我们一下子就被字符串 classes%zu.dex 所吸引,MultiDex当初在编译时生成的多个dexclasses2.dex、classes3.dex…全部都在这里被识别出来了!

  绕了那么大一个圈子,ART模式下内建MultiDex的实现到这里就算是全部弄清楚了!

原文链接:http://www.apkbus.com/blog-705730-61784.html

点击查看更多内容
TA 点赞

若觉得本文不错,就分享一下吧!

评论

作者其他优质文章

正在加载中
  • 推荐
  • 评论
  • 收藏
  • 共同学习,写下你的评论
感谢您的支持,我会继续努力的~
扫码打赏,你说多少就多少
赞赏金额会直接到老师账户
支付方式
打开微信扫一扫,即可进行扫码打赏哦
今天注册有机会得

100积分直接送

付费专栏免费学

大额优惠券免费领

立即参与 放弃机会
意见反馈 帮助中心 APP下载
官方微信

举报

0/150
提交
取消